Tags

Google PerfkitBenchmarker v1.14.0 from upstream.

PerfKitBenchmarker release for touchstone 2.11.0

Release v1.12.0

Release 1.11.0.

External contributions:
- Add ability to publish to InfluxDB (thanks @besanradwan GH-1278)
- Always set internal IP-Address on OpenStack VMs (thanks @yremmet
GH-1228)
- Add CloudSuite data-analytics benchmark (thanks @ivonindza GH-1285)
- Update README.md (thanks @ianmaddox GH-1291)
- Fix a small typo in the anti-affinity FLAG lookup (thanks @joelio
GH-1284)
- Deprecate flag openstack_volume_size (thanks @vhe182 GH-1220)
- Add ProfitBricks cloud api v3 support (thanks @aajdinov GH-1264)
- Enable OpenStack Cinder Volume Type Option (thanks @rossmartyn04
GH-1197)

New features:
- Add dpb_wordcount_benchmark (GH-1253)

Enhancements:
- Add helpmatch usage instructions (GH-1268)
- Add openStack_volume_type option decoder (GH-1267)
- Upgrade OpenStack CLI dependency to 3.6.0 (GH-1265)
- Add object storage service cold reads support, various improvements
(GH-1290, GH-1274)
- Add support for AWS Spot Instances (GH-1288)
- Bump fio version to v2.17 and enable histogram (GH-1300)
- Let publisher.py republish already-collected samples (GH 1299)
- Allow specifying temp dir (GH-1277)
- Add --json_write_mode (GH-1276)
- Allow override hpcc binaries (GH-1269)

Bug fixes and maintenance updates:
- Lock samples json file before writing to it (GH-1265)
- Fixed exception caused by publishers being None (GH-1302)
- Add error and warning log level options to reduce log size (GH-1301)
- Use force for DigitalOcean deletes (GH-1281)
- Print helpmatch usage instructions (GH-1280)
- Fix fio parser regex (GH-1275)
- Specify AWS region when calling describe-images (GH 1273)
- Make CheckPrerequisites more comprehensive (GH-1272)
- Use image_project flag in disk creation if set (GH-1271)
- Minor bug fixes and configuration improvements (GH-1262)

PerfkitBenchmarker v1.6.0

Perfkit Benchmarker version 1.4

v1.10.0

External contributions:
- Re-write flag_util_test.py so it's easier to read (thanks @yilei GH-1244)
- Fix bug that prevents running bonnie++ on Azure (thanks @rgodha GH-1218)
- Add kernel compile benchmark (thanks @maggiejiao GH-1215, GH-1255)
- Change the documentation to reflect that work should be on master (thanks
  @besanradwan GH-1209)
- Add flags to capture latency, bandwidth and IOPS logs in Fio benchmark (thanks @meteorfox GH-1205)
- Re-add 'vm_util' object to fix broken master branch (thanks @vhe182 GH-1196)
- Add EPEL Repo to Silo Installation (thanks @akrzos GH-1153)

New features:
- Add GPU PCIe bandwidth test (GH-1234, GH-1250, GH-1252, GH-1254)
- Add blazemark (GH-1145)
- Add simple memcached ycsb benchmark based on aerospike ycsb benchmark (GH-1199)
- Add scimark2 installation package (GH-1185)

Enhancements:
- Use FLAGS.aws_user_name if set for Rhel (GH-1249)
- Fio enhancements (GH-1246)
- Use aes128-ctr if aes128-cbc isn't available (GH-1242)
- Make aeropsike able to run with raw devices on static vms (GH-1243)
- Add some metadata to netperf samples (GH-1236)
- Remove unneeded Azure code and add availability sets (GH-1232)
- Add cassandra read concurrency flag (GH-1233)
- Add a metadata attribute 'ssd_count' to spark_service (GH-1229)
- Add a metadata attribute 'spark_svc_cloud' to spark_service (GH-1226)
- Add ability to specify boot_disk_size in AWS vm spec (GH-1231)
- Add a metadata attribute 'spark_svc_cloud' (GH-1236)
- Set a default zone for the spark test to allow for subnet creation (GH-1222)
- Update the location of the spark examples jar on the emr cluster (GH-1222)
- Persist package installation across reboots (GH-1185)
- Allow mongodb_ycsb to specify readahead settings (GH-1256)

Bug fixes and maintenance updates:
- Close Aersospike connection in 1 second (GH-1241)
- Fix bug with having multiple Azure data disks (GH-1223)
- Fix aws spark service (GH-1222)
- Add some git ignore entries (GH-1222)
- Fix gcs credential conflict (GH-1221)
- Fix bug with run_processes (GH-1212)
- Remove collector from RunBenchmarkTask args (GH-1210)
- Add the minimum tox version to the "tox is not installed" message (GH-1201)
- Move sysbench05plus path prefix logic into package (GH-1185)
- Fix bug with specifying Azure image (GH-1204)
- Fix two files that were failing boilerplate check (GH-1203)
- Update README.md (GH-1200)

Release 1.9.0

Changes in this release.

New features:
- Add config file imports and allow for circular imports and add documentation (GH-1163)
- Add a way to run benchmarks in parallel (GH-1192)

Enhancements:
- ContainerizedDebianMixin: Ping to a fixed docker image and remove the sudo hack (GH-1171)
- Auto detect  openjdk, libsnappy package versions (GH-1181)
- Add flags to govern subnet creation and Optionally create only 1 subnet (GH-1182)
- Install docker images as packages (GH-1184)
- Call FLAGS.get_help if it's available (GH-1187)
- Publish boot time samples for all tests (GH-1156)

Bugfixes and maintenance updates:
- Update requirements.txt to set version of contextlib2 (GH-1164)
- Fix bug with config flags (GH-1165)
- Updated the cloudsuite web-serving benchmark (thanks @nooshin-mirzadeh, GH-1166)
- Fix load command parsing (GH-1168)
- openstack: Open all TCP and UDP ports for the internal network (GH-1169)
- Fix aerospike metadata (GH-1170)
- Adjust aerospike default replication factor (GH-1172)
- Try replacing FlagValues._flags instead of FlagValues.FlagDict first (GH-1175)
- Fix ycsb aggregator (GH-1176)
- Fix bug during cleanup phase of object_storage_service benchmark (GH-1178)
- Netperf thinktime fixes and support think time in nanoseconds instead of microseconds (GH-1179, GH-1186, GH-1188)
- Fix sysbench05plus installation on ubuntu16.04 (GH-1183)
- Fix race condition in Aerospike benchmark (GH-1190)
- Clean up some pickling, unpickling issues (GH-1191)
- Calculate free ram with more robust /proc/meminfo instead of `free`(GH-1194)
- Fix multithread netperf (GH-1149)
- Multiple fixes and refactoring of the linux package management on vms (GH-1152)
- Fix dstat metadata in Analyze (GH-1154)
- Obey flag overrides for static vm specs (GH-1155)
- Fix links in README file for OpenStack and Cloudstack setup steps (thanks @shakhat, GH-1157)
- Fix issue with GetConfig using flags (GH-1160)
- Remove scratch disk from netperf and object_storage_service benchmark (GH-1147, GH-1148)
- Netperf changes to add num_streams metadata to samples (GH-1177)
- Remove collector from RunBenchmarkTask args (GH-1210)

Bugfixes and maintanence updates:
 - Fixed default machine type for Azure (GH-1161)

Release v1.8.0

New features:
- Multithreaded netperf benchmark (GH-1141, GH-1144)
- Terasort benchmark using spark service infrastructure (GH-1104)
- Support 10 or more disks on Azure (GH-1102)
- Add sysctl and set_files flags (GH-1096)
- Support DigitalOcean block storage (GH-1092)
- Add cloudsuite_data_serving_rec_count and cloudsuite_data_serving_op_count parameters to cloudsuite_data_serving benchmark (GH-1091)
- Add support for setting GCE boot disk size and type (GH-1106)
- Add flag to skip reload on ycsb benchmark (GH-1108)
- Support for AWS dedicated hosts (GH-1109)
- Object storage multistream write-only scenario (GH-1123)
- Allow multiple configs for the same benchmark in the same file (GH-1126)
- Add flat that allows run retries (GH-1129)
- Add fio_parameters flag (GH-1131)
- Add config matrices (GH-1134)
- Add events at the beginning and end of benchmarks (GH-1135)
- Add extra_zones flag to pkb (GH-1137)
- Add flag matrix filters (GH-1138)
- Support allowing port ranges on Firewall.AllowPort (GH-1139)
- Add static vm tags (GH-1142)

Breaking changes:
- Because of GH-1112, region names have changed.

Enhancements:
- Allow speccpu to run individual test (GH-1113)
- Use Azure resource management mode (GH-1112)
- GCP networks use subnets (GH-1087)
- Add top level flags key to configs (GH-1125)
- Precise resource usage for YCSB benchmarks (GH-1127)

Bugfixes and maintanence updates:
- Fix bugs with Terasort (GH-1136)
- Don't run container commands as sudo with ContainerizedDebianMixin (GH-1130)
- Automatically use amazon linux if os_type is rhel (GH-1124)
- Removed Panda as a dependency (GH-1118, GH-1119)
- Allow flag_util.ParseKeyValuePairs values to have colons in them (GH-1117)
- Add ycsb to data directory (GH-1116)
- Fix bug in mongodb_ycsb_benchmark cleanup (GH-1115)
- Fix bug where static VM disk specs were overriden (GH-1105)
- Use string formatting instead of addition to concatenate (GH-1103)
- Fixed regression with fio flag (GH-1097)
- Fix flags in static vm files (GH-1093)
- Fix wait_for_command.py: wait for status file to avoid race condition (GH-1090)

External contributions:
* Add RHEL based virtual machines to OpenStack provider. (thanks @akrzos;
  GH-858)
  * Change rackspace provider information to use official CLI (thanks @meteorfox; GH-844)
  * Add rackspace requirements (thanks @meteorfox; GH-805)

New features:
  * Support flags in YAML format. (GH-857)
  * Check version of required packages at runtime (GH-834)
  * User-specified multiregion for GCS benchmarking (GH-845)
  * Support metadata for gcp instances (GH-859)

Bugfixes and maintences updates:
  * Change rackspace's AllowPort to enable UDP (thanks @meteorfox; GH-805)
  * Allow most recent verison gcs-oauth2-boto-plugin (GH-849)
  * Require Pint >= 0.7 (GH-850)
  * Update PIP (GH-842)
  * Fix windows log message (GH-832)
  * Properly Pickle Pint Quantities (GH-830)
  * os_type added to boot benchmark metadata (GH-826)
  * Better handle Azure timeouts (GH-825)
  * Better handling of AWS integration tests. (GH-869, GH-868)

Release 1.2.0

New features:
* Introduce Object Sizes (GH-808).
* Add ListDecoder for verifying a config option that expects list values
  (GH-807).

Enhancements:
* Bump HBase to 1.0.3 (GH-822).
* Change MockFlags to be more like the real FlagValues (GH-812).
* Rename test_flag_util.py and add FlagDictSubstitution tests (GH-811).
* Create BenchmarkConfigSpec to aggregate benchmark input
  checking (GH-810).
* Remove flag proxy objects (GH-802).

Bugfixes and maintenance updates:
* Fix sample iperf sample (GH-801).
* FixIntDecoder and FloatDecoder behavior when min=0 or max=0 (GH-800).

Release 1.1.0

External contributions:
* Add additional percentile samples to mysql_service benchmark and others
  (thanks @zbjornson; GH-729).
* Add Mesos-Marathon as a provider (thanks @mateusz-blaszkowski; GH-679,
  GH-731).
* Use only available floating-ips for OpenStack (thanks @meteorfox; GH-733).
* Add OpenStack CLI-based support for object_storage_service benchmark (thanks
  @meteorfox; GH-738).
* Make printed benchmark results more readable (thanks @wonderfly; GH-747).
* Clean up formatting, grammar, and markdown in README.md (thanks @mbrukman;
  GH-748).
* Add links to projects and licenses in README.md (thanks @mbrukman; GH-763).
* Fix a bug that caused --helpxml to not work (thanks @jldiaz; GH-782).
* Make OpenStack VM username configurable with a new --openstack_image_username
  flag (thanks @lwatta; GH-788).

New features:
* Allow users to specify a storage region for the object_storage_service
  benchmark with a new --object_storage_region flag (GH-609).
* Add support for running a background CPU workload while executing a benchmark
  (GH-715, GH-762).
* Allow creating GCE custom VMs easily from command-line flags (GH-727).
* Add the --gce_network_name to allow GCE users to use pre-existing networks
  (GH-746).
* Publish a PTRANS sample from the hpcc benchmark (GH-785).
* Add support for running a background network workload while executing a
  benchmark (GH-786).
* Add a CSV sample output format (GH-791).

Enhancements:
* Trim blocks while formatting and mounting disks (GH-692).
* Raise an error when no requests succeed in redis_benchmark (GH-702).
* Update the Google benchmark set (GH-705).
* Add input-checking to disk specs (GH-675, GH-779).
* Let disks on Windows machines return Windows DeviceIds (GH-743).
* Speed up Cassandra cluster boot by setting "auto_bootstrap" to false (GH-751).
* Defer package update for Ubuntu (GH-752).
* Add provider-specific requirements.txt files (GH-753).
* Increase preload thread count for YCSB benchmarks (GH-760). This change
  affects some of the samples generated by the aerospike_ycsb, cassandra_ycsb,
  and mongodb_ycsb benchmarks, including the overall Runtime and overall
  Throughput.
* Guarantee order of VM and disk spec config option decoding (GH-792).

Bugfixes and maintenance updates:
* Create one AWS VPC/IGW/PlacementGroup per region (GH-687, GH-798).
* Update README.md instructions for setting up PKB for GCE (GH-689).
* Update comments in pkb.py to reflect new provision and teardown stages
  (GH-691).
* Fix bug that prevented object_storage_service benchmark from running in stages
  (GH-694).
* Add test helpers that make it easier to mock command-line flags (GH-695,
  GH-730).
* Delete set-interrupts.sh (GH-703).
* Make process exit status non-zero upon benchmark failure (GH-713).
* Move "To run again" message from teardown to cleanup stage (GH-714).
* Fix creation of AWS VMs using Placement Groups (GH-721).
* Create AWS/GCE default access rules before creating VMs (GH-726, GH-756).
  This change affects the Boot Time measured by the cluster_boot benchmark.
* Fix a GCE boot disk creation bug (GH-736).
* ntttcp benchmark fixes (GH-737).
* Force mongodb_ycsb benchmark to use only one VM (GH-755).
* Unmount /mnt if it exists prior to creating a scratch filesystem (GH-757).
* Move provider constants to a separate module (GH-758).
* Fix a bug when creating an AWS VPC outside the user's default region (GH-774).
* Add tests for generating help strings (GH-782).
* Kill servers started by the iperf benchmark as part of its cleanup (GH-784).
* Fix some style inconsistencies (GH-789).
* Move operating system constants to a separate module (GH-790).
* Fix some broken tests (GH-795, GH-796).