commit f5b1f1fd7c0be0506ba73502a675de9d048b744e Author: Jenkins Build Slave User Date: Thu Jul 6 14:56:18 2017 +0000 10.2.8 commit 66dbf9beef04988dbd3653591e51afa6d84e3990 Merge: 2f491b2e5e f46ccf2cb4 Author: Nathan Cutler Date: Tue Jul 4 17:43:57 2017 +0200 Merge pull request #14710 from smithfarm/wip-start-race tests: rados: sleep before ceph tell osd.0 flush_pg_stats after restart Reviewed-by: Sage Weil Reviewed-by: Kefu Chai Reviewed-by: David Zafman commit 2f491b2e5e8b1f340b28415c3bd3d9628603c377 Merge: 552a573f84 a372b4eca1 Author: Yuri Weinstein Date: Tue Jul 4 07:31:50 2017 -0700 Merge pull request #16089 from ceph/wip_fix_point_jewel qa/Fixed upgrade sequence to 10.2.0 -> 10.2.7 -> latest -x (10.2.8) Reviewed-by: Nathan Cutler commit 552a573f8426ecfec1a0df21a6c3941afd4e460c Merge: 53a3be7261 55eeaadfc4 Author: Yuri Weinstein Date: Mon Jul 3 17:23:43 2017 -0700 Merge pull request #16088 from smithfarm/wip-fix-client-upgrade-centos tests: run upgrade/client-upgrade on latest CentOS 7.3 Reviewed-by: Yuri Weinstein commit a372b4eca1f25647541943918ae737f20783db11 Author: Yuri Weinstein Date: Mon Jul 3 14:18:14 2017 -0700 Fixed upgrade sequence to 10.2.0 -> 10.2.7 -> latest -x (10.2.8) Signed-off-by: Yuri Weinstein commit 55eeaadfc4025c83cb63c951265710868df0325f Author: Nathan Cutler Date: Mon Jul 3 22:55:21 2017 +0200 tests: run upgrade/client-upgrade on latest CentOS 7.3 Before this patch, all centos jobs were failing because there are no longer any CentOS 7.2 machines in Sepia. Signed-off-by: Nathan Cutler commit 53a3be7261cfeb12445fbdba8238eefa40ed09f5 Merge: 84bd162978 d33b30cdb0 Author: Nathan Cutler Date: Fri Jun 30 16:43:38 2017 +0200 Merge pull request #15504 from Vicente-Cheng/wip-20151-jewel jewel: ceph-disk: do not setup_statedir on trigger Reviewed-by: Josh Durgin commit 84bd162978e48eead40335bcbd92e4ab18e9c590 Merge: c710689109 8e0e4a0ce7 Author: Nathan Cutler Date: Wed Jun 28 10:20:23 2017 +0200 Merge pull request #15904 from smithfarm/wip-20413-jewel jewel: tests: upgrade:hammer-x/stress-split-erasure-code-x86_64 fails in 10.2.8 integration testing Reviewed-by: Brad Hubbard commit c7106891096c895f0cc5c2cef438078ea48de95d Merge: 5c6cb14806 9d3110c276 Author: Nathan Cutler Date: Wed Jun 28 08:16:33 2017 +0200 Merge pull request #14930 from smithfarm/wip-19829-jewel jewel: tests: New upgrade test for #14930 Reviewed-by: Josh Durgin commit 5c6cb1480699f5ce464e25b9cacdda770ce3660d Merge: bdc085d02a d43e19d886 Author: Nathan Cutler Date: Wed Jun 28 08:16:04 2017 +0200 Merge pull request #14392 from asheplyakov/19508-jewel jewel: osd: pg_pool_t::encode(): be compatible with Hammer <= 0.94.6 Reviewed-by: Josh Durgin commit 9d3110c276917055b078cd14c181b2bda2625821 Author: Nathan Cutler Date: Sun Jun 25 10:32:16 2017 +0200 tests: upgrade/hammer-x/v0-94-6-mon-overload: tweak packages list Include some hammer dependencies that aren't in the jewel default packages list, and exclude some java packages that may not be in the hammer repo and are not needed for the upgrade test in any case. N.B.: This cannot be cherry-picked from master because upgrade/hammer-x was dropped in master. Signed-off-by: Nathan Cutler commit 6a64f8901bb3b218a8dc58b11d6c13033d45f067 Author: Nathan Cutler Date: Wed May 3 11:39:27 2017 +0200 tests: upgrade/hammer-x: new v0-94-6-mon-overload subsuite This is not a cherry-pick from master because direct upgrades from hammer to kraken+ are not supported. Fixes: http://tracker.ceph.com/issues/19829 References: http://tracker.ceph.com/issues/19508 Signed-off-by: Nathan Cutler commit bdc085d02ab9723f6b90b6a3047bc51cf224b930 Merge: e41ae4a7b1 d2d4b7202d Author: Nathan Cutler Date: Tue Jun 27 12:41:59 2017 +0200 Merge pull request #15936 from batrick/i20412 qa: enable quotas for pre-luminous quota tests Reviewed-by: John Spray Reviewed-by: Nathan Cutler commit d2d4b7202d77e5696eb18c4da4f7d614116ced36 Author: Patrick Donnelly Date: Mon Jun 26 19:04:48 2017 -0700 qa: enable quotas for pre-luminous quota tests This cannot be cherry-picked from master because the config option is removed since 0f250a889dba2100d3afcea0a18e4f6a8d086b86. Fixes: http://tracker.ceph.com/issues/20412 Signed-off-by: Patrick Donnelly commit e41ae4a7b1b9cc4394473e21f6e6d6ef9cab1d59 Merge: a21af3b7a1 682b4d717c Author: Sage Weil Date: Mon Jun 26 21:24:11 2017 -0500 Merge pull request #15933 from smithfarm/wip-hammer-jewel-x jewel: tests: drop upgrade/hammer-jewel-x commit 682b4d717c96b516c315a01b1174af3503dedba6 Author: Nathan Cutler Date: Tue Jun 27 02:27:22 2017 +0200 tests: drop upgrade/hammer-jewel-x This suite doesn't have any test logic in it. Its existence in the jewel branch appears to be an oversight. This cannot be cherry-picked from master because the upgrade/hammer-jewel-x suite is present (and justified) in master and is not currently being dropped there. Signed-off-by: Nathan Cutler commit a21af3b7a1ea5abfa4f344800e413d6249824204 Merge: 615a6ab9b7 a744340790 Author: John Spray Date: Mon Jun 26 16:25:12 2017 -0400 Merge pull request #15438 from Vicente-Cheng/wip-20027-jewel jewel: mds: issue new caps when sending reply to client Reviewed-by: John Spray commit 615a6ab9b723d204c6de567750fe9450742fcedb Merge: 9b13b48b3e 8ac0e5c363 Author: John Spray Date: Mon Jun 26 16:23:48 2017 -0400 Merge pull request #15000 from jan--f/wip-19846-jewel jewel: cephfs: normalize file open flags internally used by cephfs Reviewed-by: John Spray commit 9b13b48b3ed919340789a41d065eb4a9a27110de Merge: d217da1742 b429fa1807 Author: Nathan Cutler Date: Mon Jun 26 18:25:03 2017 +0200 Merge pull request #15383 from asheplyakov/20014-bp-jewel jewel: cls/rgw: list_plain_entries() stops before bi_log entries Reviewed-by: Orit Wasserman commit d217da174252f9126d530868aa472230ecba31ca Merge: e520040ece 4028774122 Author: Zack Cerza Date: Mon Jun 26 10:17:03 2017 -0600 Merge pull request #15870 from smithfarm/wip-swift-task-move-jewel tests: move swift.py task from teuthology to ceph, phase one (jewel) commit e520040ecec756ce181f716dad1c0bad41c77a7d Merge: dde8656e6b 1c0c9093ab Author: Nathan Cutler Date: Mon Jun 26 09:24:04 2017 +0200 Merge pull request #15842 from smithfarm/wip-sortbitwise-jewel qa/suites/upgrade/hammer-x: set "sortbitwise" for jewel clusters Reviewed-by: Kefu Chai commit dde8656e6b5e7ffe66a6fd695cbc17dfb18fb43e Merge: 498c96e66a 06cf9f3650 Author: John Spray Date: Sun Jun 25 19:59:40 2017 -0400 Merge pull request #15468 from smithfarm/wip-20140-jewel jewel: cephfs: Journaler may execute on_safe contexts prematurely Reviewed-by: John Spray commit 4028774122954023265d7825fbf9e91dc526fdee Author: Nathan Cutler Date: Sun Jun 25 12:42:36 2017 +0200 tests: swift.py: tweak imports The ".." form only works within the teuthology repo. With swift.py now in the Ceph repo, we have to be explicit. Error message was: "ValueError: Attempted relative import beyond toplevel package Signed-off-by: Nathan Cutler commit 8e0e4a0ce7489542f47522e0a5161a5bf123c744 Author: Nathan Cutler Date: Sun Jun 25 10:27:58 2017 +0200 tests: upgrade/hammer-x/stress-split: tweak packages list Include some hammer dependencies that aren't in the jewel default packages list, and exclude some java packages that may not be in the hammer repo and are not needed for the upgrade test in any case. N.B.: This cannot be cherry-picked from master because upgrade/hammer-x was dropped in master. Signed-off-by: Nathan Cutler commit a86ce728954a765797ce634025d43650d990e480 Author: Nathan Cutler Date: Sun Jun 25 09:27:47 2017 +0200 tests: swift.py: clone the ceph-jewel branch The master branch of ceph/swift.git contains tests that are incompatible with Jewel and Hammer. The ceph-jewel branch omits these tests. Signed-off-by: Nathan Cutler commit 498c96e66a91edc8bd614cfc8fc5a14b3d210a76 Merge: 38af498f9f cda721bbbf Author: John Spray Date: Fri Jun 23 08:02:48 2017 -0400 Merge pull request #15472 from smithfarm/wip-20148-jewel jewel: mds: Too many stat ops when trying to probe a large file Reviewed-by: John Spray commit 3d5b489369bb2cecccb1f36347654c0a37069d1c Merge: 38af498f9f 7b58ac97e9 Author: Nathan Cutler Date: Fri Jun 23 08:35:27 2017 +0200 Merge branch 'master' of /home/smithfarm/src/ceph/upstream/teuthology into wip-swift-task-move-jewel commit 7b58ac97e9dd195f4170e9e0ea00bae76d1f3ccd Author: Nathan Cutler Date: Fri Jun 23 08:27:42 2017 +0200 tests: move swift.py task to qa/tasks In preparation for moving this task from ceph/teuthology.git into ceph/ceph.git The move is necessary because jewel-specific changes are needed, yet teuthology does not maintain a separate branch for jewel. Also, swift.py is a Ceph-specific task so it makes more sense to have it in Ceph. Signed-off-by: Nathan Cutler commit 38af498f9f7b62f9f851364ae7f2691832423198 Merge: d0ae1de51f aa0cd461df Author: Nathan Cutler Date: Thu Jun 22 22:07:23 2017 +0200 Merge pull request #15529 from badone/wip-async-sleep-timer-fix-jewel jewel: osd: Implement asynchronous scrub sleep Reviewed-by: Josh Durgin commit 1c0c9093ab913a82c1dc5656a54b4009bdc35c9c Author: Nathan Cutler Date: Thu Jun 22 11:32:42 2017 +0200 qa/suites/upgrade/hammer-x: set "sortbitwise" for jewel clusters Inspired by 3734280522a913ca8340ebc98b80978f63bade6f This cannot be cherry-picked from master because master does not have qa/suites/upgrade/hammer-x Signed-off-by: Nathan Cutler commit d0ae1de51f5faf26a2f4b0d5b7f494a4923f870d Merge: 64c011a8c4 de76fdbb9f Author: Kefu Chai Date: Thu Jun 22 11:59:51 2017 +0800 Merge pull request #15824 from tchaikov/jewel qa/workunits/rados/test-upgrade-*: whitelist tests the right way Reviewed-by: Sage Weil commit de76fdbb9f435652e2c15326d00b01d26ab007a7 Author: Kefu Chai Date: Thu Jun 22 08:06:43 2017 +0800 qa/workunits/rados/test-upgrade-*: whitelist tests the right way --gtest_filter=POSTIVE_PATTERNS[-NEGATIVE_PATTERNS], so we cannot add multiple exclusive patterns using -pattern:-pattern, instead, we should use: -pattern:pattern Signed-off-by: Kefu Chai Conflicts: qa/workunits/rados/test-upgrade-v11.0.0.sh: this change is not cherry-picked from master, because the clone-range op was removed from master. and only supported in pre-luminous releases. commit 64c011a8c4af27dc095b1a9190ccf1ca76d2cc8f Merge: e8da5e376f ab78cd040f Author: Nathan Cutler Date: Tue Jun 20 22:54:00 2017 +0200 Merge pull request #14661 from smithfarm/wip-19575-jewel jewel: rgw: unsafe access in RGWListBucket_ObjStore_SWIFT::send_response() Reviewed-by: Casey Bodley commit e8da5e376fc426f85fcab84a5ae71b3c17ed0068 Merge: 5a1e849ecf 1af6781d3c Author: Kefu Chai Date: Tue Jun 20 22:42:47 2017 +0800 Merge pull request #15778 from tchaikov/wip-upgrade-without-clone-range-jewel qa/workunits/rados/test-upgrade-*: whitelist tests for master Reviewed-by: Sage Weil commit d43e19d88692bd318f0569559867df919c26d8db Author: Alexey Sheplyakov Date: Fri Apr 7 12:34:20 2017 +0400 jewel: osd: pg_pool_t::encode(): be compatible with Hammer <= 0.94.6 This patch is necessary for Jewel only since direct upgrades from Hammer to Kraken and newer are not supported. Fixes: http://tracker.ceph.com/issues/19508 Signed-off-by: Alexey Sheplyakov commit 1af6781d3c60421930087d31124e62cae530ca24 Author: Kefu Chai Date: Tue Jun 20 19:49:14 2017 +0800 qa/workunits/rados/test-upgrade-*: whitelist tests for master The jewel-x upgrade test now runs this script against a mixed cluster on a machine with code from master installed. That means we have to skip any new tests that will fail on a mixed cluster. CloneRange was removed in 0d7b0b7. Signed-off-by: Kefu Chai Conflicts: qa/workunits/rados/test-upgrade-v11.0.0.sh: this change is not cherry-picked from master, because the clone-range op was removed from master. and only supported in pre-luminous releases. commit a74434079088129244b7aae6ccc6df7094282eba Author: Yan, Zheng Date: Sat Apr 22 12:27:12 2017 +0800 mds: issue new caps when sending reply to client After Locker::issue_new_caps() adds new Capability data struct, do not issue caps immediately. Let CInode::encode_inodestate() do the job instead. This can avoid various races that early reply is not allowed, caps that haven't been sent to client gets revoked. Fixes: http://tracker.ceph.com/issues/19635 Signed-off-by: "Yan, Zheng" (cherry picked from commit 799703a4acb49db0b6cc99a23e4326767e694c3a) commit d33b30cdb019937ff88f9724599f52f4e00d37cf Author: Loic Dachary Date: Thu Jun 1 11:37:20 2017 +0200 ceph-disk: do not setup_statedir on trigger trigger may run when statedir is unavailable and does not use it. Fixes: http://tracker.ceph.com/issues/19941 Signed-off-by: Loic Dachary (cherry picked from commit 16bfbdd3d9988523bba31aace516c303057daa58) commit f46ccf2cb4701cd93cd9b15a4e57b5b97798b947 Author: Nathan Cutler Date: Fri Apr 21 11:05:05 2017 +0200 tests: rados: sleep before ceph tell osd.0 flush_pg_stats after restart Even though we wait for HEALTH_OK after restarting the daemons, they are not ready to respond to flush_pg_stats. The reason why the osd is not ready for "tell" command after "ceph health" shows that the cluster is "HEALTH_OK" is that the monitor fails to be notified that the osd in question is not up in "heatbeat_interval". Because infernalis does not have the osd_fast_fail_on_connection_refused support, the monitor needs longer to detect that an osd is down, and osd_heartbeat_grace is used to determine if an osd is down. References: http://tracker.ceph.com/issues/16239 Signed-off-by: Nathan Cutler Signed-off-by: Kefu Chai commit ab78cd040f6d3946ed40b6638ebcf52969a7cbb6 Author: Yehuda Sadeh Date: Wed Mar 8 14:52:34 2017 -0800 rgw: fix crash when listing objects via swift Fixes: http://tracker.ceph.com/issues/19249 Signed-off-by: Yehuda Sadeh (cherry picked from commit a9ec5e8ce184e19c009863db4d3519f9d8af91bd) Conflicts: src/rgw/rgw_rest_swift.cc ("key" element of RGWObjEnt struct is not a reference; fix) (cherry picked from commit 92b35155ff7b7492f3c50bf4f2ff0ffef2ca1c55) commit 5a1e849ecf215d82e31b9bdd0970cb04200de2c9 Merge: 2469085d57 66c3db7aee Author: Nathan Cutler Date: Mon Jun 19 22:46:47 2017 +0200 Merge pull request #14752 from cbodley/wip-19474 jewel: rgw: allow system users to read SLO parts Reviewed-by: Casey Bodley commit cda721bbbfae00ec4244718ae20cbd9ae914c630 Author: Yan, Zheng Date: Fri May 19 09:37:15 2017 +0800 client: update the 'approaching max_size' code The old 'approaching max_size' code expects MDS set max_size to '2 x reported_size'. This is no longer true. The new code reports file size when half of previous max_size increment has been used. Signed-off-by: "Yan, Zheng" (cherry picked from commit 9316b0442c6f828dcf8da952e4c7a63c4db1398d) Conflicts: src/client/Client.cc - in jewel, second argument to check_caps() is a bool (see 0df562a8e13 which is not in jewel) commit 439f39128ec278ce78139d4b96ed098c68efa3f5 Author: Yan, Zheng Date: Wed May 17 19:08:37 2017 +0800 mds: limit client writable range increment For very large file, setting the writable range to '2 * file_size' causes file recovery to run a long time. To recover a 1T file, Filer needs to probe 2T~1T range. Fixes: http://tracker.ceph.com/issues/19955 Signed-off-by: "Yan, Zheng" (cherry picked from commit 538f35bef944b18e9bca2b15ed7f4e8807ef0554) Conflicts: src/mds/Locker.h - in jewel, file_update_finish() has different arguments than it does in master commit 06cf9f365033f7913051bdf4060f0bc6fc0444d7 Author: Yan, Zheng Date: Tue May 23 21:46:54 2017 +0800 osdc/Journaler: avoid executing on_safe contexts prematurely Journaler::_do_flush() can skip flushing some data when prezered journal space isn't enough. Before updating Journaler::next_safe_pos, we need to check if Journaler::_do_flush() has flushed enough data. Fixes: http://tracker.ceph.com/issues/20055 Signed-off-by: "Yan, Zheng" (cherry picked from commit 6511e7a9e35a14216c03cd6921ca4d232274f953) commit 2e299b50de4a297fee2aec21290632336d239857 Author: Yan, Zheng Date: Wed Apr 12 16:00:18 2017 +0800 osdc/Journaler: make header write_pos align to boundary of flushed entry This can speed up the process that detects and drops partial written entry in the log tail. Signed-off-by: "Yan, Zheng" (cherry picked from commit 8ae2962b79903e217fda83cea4140af64b5d6883) Conflicts: src/osdc/Journaler.cc - 8d4f6b92cba is not being backported to jewel src/osdc/Journaler.h - Journaler::Journaler initializer list is different in jewel, compared to master commit 2469085d57a05933589165f6f99a67b2e28c7022 Merge: 42c3fbc129 d57437e338 Author: John Spray Date: Wed Jun 14 10:01:27 2017 -0400 Merge pull request #14672 from smithfarm/wip-19334-jewel jewel: MDS heartbeat timeout during rejoin, when working with large amount of caps/inodes Reviewed-by: John Spray commit 8ac0e5c363bd6439071d26874b6714cf2376736f Author: Yan, Zheng Date: Fri May 12 10:38:51 2017 +0800 pybind: fix cephfs.OSError initialization Traceback (most recent call last): File "", line 1, in File "cephfs.pyx", line 672, in cephfs.LibCephFS.open (/home/zhyan/Ceph/ceph-2/build/src/pybind/cephfs/pyrex/cephfs.c:10160) File "cephfs.pyx", line 155, in cephfs.OSError.__init__ (/home/zhyan/Ceph/ceph-2/build/src/pybind/cephfs/pyrex/cephfs.c:1889) TypeError: __init__() takes exactly 3 positional arguments (2 given) Signed-off-by: "Yan, Zheng" (cherry picked from commit e6493f64ba4592b8dca54ece4464efa6c7f331a7) commit 09b9410c2e69a466b001d92fc14eb44d768009f1 Author: Yan, Zheng Date: Wed May 10 08:13:52 2017 +0800 pybind: fix open flags calculation (O_WRONLY | O_RDWR) is invaild open flags Fixes: http://tracker.ceph.com/issues/19890 Signed-off-by: "Yan, Zheng" (cherry picked from commit 2c25c99cb4572ffae97555a56b24a4c4097dcdec) commit 42c3fbc129cbb60d447c1a6b0402a9def1656446 Merge: 71d45e1905 7347f11939 Author: John Spray Date: Wed Jun 14 09:43:05 2017 -0400 Merge pull request #14677 from smithfarm/wip-19665-jewel jewel: mds: C_MDSInternalNoop::complete doesn't free itself Reviewed-by: John Spray commit 71d45e190528124a8ff7e4674f7bfb7c340f80ee Merge: 27c915f5d2 e6daee8a9f Author: John Spray Date: Wed Jun 14 09:35:08 2017 -0400 Merge pull request #15466 from smithfarm/wip-19762-jewel jewel: cephfs: non-local quota changes not visible until some IO is done Reviewed-by: John Spray commit 27c915f5d21bd84502e35eb269d955fafc47de0b Merge: a76357622a db053da618 Author: John Spray Date: Wed Jun 14 09:32:47 2017 -0400 Merge pull request #14700 from smithfarm/wip-19709-jewel jewel: mds: enable start when session ino info is corrupt Reviewed-by: John Spray commit a76357622a2773850153aa1f6ea02b1737942c4b Merge: 85aab833f7 db86a24e79 Author: John Spray Date: Wed Jun 14 09:31:22 2017 -0400 Merge pull request #14685 from smithfarm/wip-19675-jewel jewel: cephfs: Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient) Reviewed-by: John Spray commit 85aab833f7f66ac81d8b12f4203fa215787f8d2a Merge: f6b395115e 7b9283beec Author: John Spray Date: Wed Jun 14 09:30:53 2017 -0400 Merge pull request #14684 from smithfarm/wip-19673-jewel jewel: cephfs: mds is crushed, after I set about 400 64KB xattr kv pairs to a file Reviewed-by: John Spray commit f6b395115e4de15d73269ff6b96f2ee0fd0ea9c3 Merge: ff60fceb5f b52c508861 Author: John Spray Date: Wed Jun 14 09:30:13 2017 -0400 Merge pull request #14683 from smithfarm/wip-19671-jewel jewel: cephfs: MDS assert failed when shutting down Reviewed-by: John Spray commit ff60fceb5f5f7f9df4a48da4ad8bd863b9a04e2d Merge: 6a6d57d2de 96e801fb53 Author: John Spray Date: Wed Jun 14 09:29:50 2017 -0400 Merge pull request #14682 from smithfarm/wip-19668-jewel jewel: cephfs: MDS goes readonly writing backtrace for a file whose data pool has been removed Reviewed-by: John Spray commit 6a6d57d2de674c5a971e999cb2731b2d8ae1b523 Merge: 8260669efa f34489dd52 Author: John Spray Date: Wed Jun 14 09:27:13 2017 -0400 Merge pull request #14679 from smithfarm/wip-19666-jewel jewel: cephfs: The mount point break off when mds switch hanppened. Reviewed-by: John Spray commit 8260669efa74f41d3b1b1039fb7b34e070951c34 Merge: 013529b61f 824b19a9a6 Author: John Spray Date: Wed Jun 14 09:26:40 2017 -0400 Merge pull request #14676 from smithfarm/wip-19619-jewel jewel: cephfs: MDS server crashes due to inconsistent metadata. Reviewed-by: John Spray commit 013529b61fc4fbf02656f7c6cb0baa1bc6004758 Merge: b518522f64 eab56dae67 Author: John Spray Date: Wed Jun 14 09:24:38 2017 -0400 Merge pull request #14674 from smithfarm/wip-19482-jewel jewel: cephfs: No output for ceph mds rmfailed 0 --yes-i-really-mean-it command Reviewed-by: John Spray commit b518522f64b66f4a14618a9345b6314ca0f2c54c Merge: 388e0d1bc3 63f41d543f Author: John Spray Date: Wed Jun 14 09:23:27 2017 -0400 Merge pull request #14671 from smithfarm/wip-19044-jewel jewel: tests: buffer overflow in test LibCephFS.DirLs Reviewed-by: John Spray commit 388e0d1bc35985a6916d3eb1ca5184a0907b6e8b Merge: d7c7ce7ebb 7146816065 Author: John Spray Date: Wed Jun 14 09:23:13 2017 -0400 Merge pull request #14670 from smithfarm/wip-18949-jewel jewel: mds: avoid reusing deleted inode in StrayManager::_purge_stray_logged Reviewed-by: John Spray commit d7c7ce7ebbb663dab0dfa8058c845d494d7615cc Merge: d717ef73bc d8b139b584 Author: John Spray Date: Wed Jun 14 09:22:28 2017 -0400 Merge pull request #14669 from smithfarm/wip-18900-jewel jewel: cephfs: Test failure: test_open_inode Reviewed-by: John Spray commit d717ef73bc0d5ef24551ec9157385c8b0521380b Merge: c2a3b7567f 36c86f71ef Author: John Spray Date: Wed Jun 14 09:21:43 2017 -0400 Merge pull request #14668 from smithfarm/wip-18705-jewel jewel: mds: fragment space check can cause replayed request fail Reviewed-by: John Spray commit c2a3b7567fbe0b2f62bcd38cd9bb9a5a2a238743 Merge: fd9256b770 5b56214519 Author: John Spray Date: Tue Jun 13 19:05:55 2017 +0100 Merge pull request #14698 from smithfarm/wip-19677-jewel jewel: cephfs: ceph-fuse does not recover after lost connection to MDS Reviewed-by: John Spray commit fd9256b77010066c934fd0016eb6d3f9c1fb54e1 Merge: 26ada59c81 c49b114e8d Author: Abhishek L Date: Fri Jun 9 19:39:17 2017 +0200 Merge pull request #14766 from smithfarm/wip-19757-jewel jewel: rgw: fix failed to create bucket if a non-master zonegroup has a single zone Reviewed-by: Abhishek Lekshmanan Reviewed-by: Casey Bodley commit 26ada59c810bfda5c16f2d935a2b98711e7b2c76 Merge: fdd25c2bdb e552d91f73 Author: Abhishek L Date: Fri Jun 9 19:38:46 2017 +0200 Merge pull request #14787 from linuxbox2/jewel-rgw-shard-limit-ck jewel: rgw: add bucket size limit check to radosgw-admin Reviewed-by: Abhishek Lekshmanan Reviewed-by: Casey Bodley commit fdd25c2bdb9ec6bb8e3061088b8782bfb2331bc4 Merge: cfd6750416 fb3ee2efcc Author: Abhishek L Date: Fri Jun 9 19:38:23 2017 +0200 Merge pull request #14789 from mdw-at-linuxbox/wip-jewel-rgw-rvk jewel: rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0 Reviewed-by: Abhishek Lekshmanan Reviewed-by: Casey Bodley commit cfd6750416baeb2bbd3836bc7dc55d83fc214c20 Merge: ce1fc3492e 86980a045b Author: Abhishek L Date: Fri Jun 9 19:36:02 2017 +0200 Merge pull request #14815 from smithfarm/wip-19786-jewel jewel: rgw: failure to create s3 type subuser from admin rest api Reviewed-by: Abhishek Lekshmanan Reviewed-by: Casey Bodley commit ce1fc3492e87c669f7059c2047a3bed077418a89 Merge: 7ca0252560 aa99558934 Author: Nathan Cutler Date: Wed Jun 7 20:13:15 2017 +0200 Merge pull request #15312 from theanalyst/wip-20078 jewel: rgw: only append zonegroups to rest params if not empty Reviewed-by: Casey Bodley commit 7ca0252560d1ff23384afb50a9c2ae2aad2ce85c Merge: 62c500f522 59bd6711a4 Author: Nathan Cutler Date: Wed Jun 7 13:48:06 2017 +0200 Merge pull request #15382 from theanalyst/wip-mem-leak2 jewel: rgw:fix memory leaks in data/md sync Reviewed-by: Casey Bodley commit aa0cd461df5fee6d143bc07440ec6de829761cef Author: Brad Hubbard Date: Mon May 22 13:21:25 2017 +1000 osd: Move scrub sleep timer to osdservice PR 14886 erroneously creates a scrub sleep timer for every pg resulting in a proliferation of threads. Move the timer to the osd service so there can be only one. Fixes: http://tracker.ceph.com/issues/19986 Signed-off-by: Brad Hubbard (cherry picked from commit f110a82437df79dc20207d296e8229fc0e9ce18b) Conflicts: src/osd/PG.cc - ceph_clock_now requires a CephContext argmunent in Jewel commit c47bd0562b1187ffb0b1b2c1ef5f105aa7951d10 Author: Brad Hubbard Date: Mon Apr 24 14:10:47 2017 +1000 osd: Implement asynchronous scrub sleep Rather than blocking the main op queue just do an async sleep. Fixes: http://tracker.ceph.com/issues/19497 Signed-off-by: Brad Hubbard (cherry picked from commit 7af3e86c2e4992db35637864b83832535c94d0e6) commit e6daee8a9fbc576da2a03550a81056d093a516c9 Author: Nathan Cutler Date: Sun Jun 4 20:39:58 2017 +0200 Client.cc: adjust Client::_getattr calls Signed-off-by: Nathan Cutler commit a2c7a2262ac8ecbea78f09e6e8e6a37498568d57 Author: John Spray Date: Wed Mar 15 19:36:08 2017 +0000 qa/cephfs: use getfattr/setfattr helpers Signed-off-by: John Spray (cherry picked from commit dd43d3bc646aeab88486b0963fc83de0b18800c4) Conflicts: qa/tasks/cephfs/test_data_scan.py: difference in the self._mount.run_shell() call in NonDefaultLayout::write (which is being dropped by this commit) - in jewel it has "sudo", and in master it doesn't commit 12aa35a6d50f612df77199ac4f35c7baeed0583e Author: John Spray Date: Wed Jun 22 13:00:44 2016 +0100 tasks/cephfs: fix race while mounting This could fail if the mount hadn't finished coming up. Signed-off-by: John Spray (cherry picked from commit adfb757c898a80f18c15dafd02e29840c5931c87) commit a7b699269b65c76361fcb1d10593812be40c7612 Author: John Spray Date: Wed Mar 15 19:26:30 2017 +0000 qa: add test for reading quotas from different clients Fixes: http://tracker.ceph.com/issues/17939 Signed-off-by: John Spray (cherry picked from commit 61617f8f10a6322603a9add77980865cd972ef97) commit 8b8ee392b8093b9b140dbbe895691f69ae40440f Author: John Spray Date: Wed Mar 15 17:51:44 2017 +0000 client: _getattr on quota_root before using in statfs ...so that after someone adjusts the quota settings on an inode that another client is using as its mount root, the change is visible immediately on the other client. Signed-off-by: John Spray (cherry picked from commit 3d25941aadd223669448d0f5d3c0bd1fefa72308) commit dd7d59a08141d6a24b172c22f5e27c8962e25fb9 Author: John Spray Date: Wed Mar 15 15:32:47 2017 +0000 client: getattr before read on ceph.* xattrs Previously we were returning values for quota, layout xattrs without any kind of update -- the user just got whatever happened to be in our cache. Clearly this extra round trip has a cost, but reads of these xattrs are fairly rare, happening on admin intervention rather than in normal operation. Fixes: http://tracker.ceph.com/issues/17939 Signed-off-by: John Spray (cherry picked from commit 532dc4b68e538c189ef828f67cecd0d647a62250) commit 62c500f52240eaa5faadd3795bd9ec84bdcbc6c7 Merge: 2badc2416c 8dd93cabd5 Author: David Zafman Date: Fri Jun 2 09:54:22 2017 -0700 Merge pull request #15416 from dzafman/wip-20126 Reviewed-by: Josh Durgin commit 8dd93cabd52cbafc29a47862f343431eb6f1cfe3 Author: David Zafman Date: Wed May 31 15:39:19 2017 -0700 osd: Object level shard errors are tracked and used if no auth available Shards with object mismatch are tracked to mark them inconsistent Fix test because storing omap_digest in object_info not behaving as before Fixes: http://tracker.ceph.com/issues/20089 Signed-off-by: David Zafman (cherry picked from commit 1cacbea763c7aabfeaaf4bd5e878301044184117) Conflicts: src/test/osd/osd-scrub-repair.sh (no alloc_hint in object_info) commit 59bd6711a47c354117a612dd4ef033d70d449383 Author: weiqiaomiao Date: Wed Jun 1 17:20:49 2016 +0800 rgw:fix memory leaks Signed-off-by: weiqiaomiao (cherry picked from commit 73e5be2b6133cf4caa0e5e5c8c9eae748b785dbf) commit b429fa1807062716c9705ddcf316ed9b2741cc43 Author: Casey Bodley Date: Fri May 5 14:56:40 2017 -0400 cls/rgw: list_plain_entries() stops before bi_log entries list_plain_entries() was using encode_obj_versioned_data_key() to set its end_key, which gives a prefix of BI_BUCKET_OBJ_INSTANCE_INDEX[=2] that range between start_key and end_key would not only span the BI_BUCKET_OBJS_INDEX[=0] prefixes, but BI_BUCKET_LOG_INDEX[=1] prefixes as well. this can result in list_plain_entries() trying and failing to decode a rgw_bi_log_entry as a rgw_bucket_dir_entry Fixes: http://tracker.ceph.com/issues/19876 Signed-off-by: Casey Bodley (cherry picked from commit b29a1633a57abf443d5790c13d680d2917f86037) commit 2badc2416c9e16babbc91364502dcb03877e88bf Merge: 5d2a68eb90 1f895c2403 Author: Sage Weil Date: Tue May 30 09:39:47 2017 -0500 Merge pull request #15360 from liewegas/wip-jewel-master-mixed qa/workunits/rados/test-upgrade-*: whitelist tests for master Reviewed-by: Kefu Chai commit 1f895c24030b79ae9f2eae16b09582fc72928b6a Author: Sage Weil Date: Tue May 30 09:58:09 2017 -0400 qa/workunits/rados/test-upgrade-*: whitelist tests for master The jewel-x upgrade test now runs this script against a mixed cluster on a machine with code from master installed. That means we have to skip any new tests that will fail on a mixed cluster. Signed-off-by: Sage Weil commit 5d2a68eb903f533689d1a1cbc410ca940baeaff3 Merge: 998d1ee4f5 81e35b9416 Author: Nathan Cutler Date: Sat May 27 09:21:44 2017 +0200 Merge pull request #15208 from liewegas/wip-sortbitwise-jewel mon: fix 'sortbitwise' warning on jewel Reviewed-by: Josh Durgin commit 998d1ee4f503cb4875283fe8d2140f3b10a26643 Merge: 54bc1e13ea 99c65bbc18 Author: Nathan Cutler Date: Sat May 27 02:46:55 2017 +0200 Merge pull request #14851 from yehudasa/wip-rgw-support-ragweed-jewel jewel: rgw: add apis to support ragweed suite Reviewed-by: Nathan Cutler commit aa99558934c6143cb35e722148b87f35ccecc397 Author: Yehuda Sadeh Date: Tue Aug 23 10:22:42 2016 -0700 rgw: rest conn functions cleanup, only append zonegroup if not empty Signed-off-by: Yehuda Sadeh (cherry picked from commit 4e41af19846db75081cb0ddb7b33dc2bb9321ace) commit 5d90798432a6446636699fc03b2f548010b1212f Author: Karol Mroz Date: Thu Mar 17 10:32:14 2016 +0100 rgw: rest and http client code to use param vectors Replaces param/header lists with vectors. In these cases, we're only ever adding to the back of the list, so a vector should be more efficient. Also moves param_pair_t/param_vec_t higher up the include chain for cleaner function signatures. Signed-off-by: Karol Mroz (cherry picked from commit d4a2527872e0f5c3ae2874bb7d0ff459ae83cfd4) Conflicts: src/rgw/rgw_http_client.cc trivial ws conflict on rebase commit 54bc1e13ea19642c8d5893f29e96ee5c053f9c59 Merge: 966f222917 43327f83ef Author: Alfredo Deza Date: Thu May 25 12:45:30 2017 -0400 Merge pull request #14765 from smithfarm/wip-18972-jewel jewel: ceph-disk does not support cluster names different than 'ceph' Reviewed-by: Alfredo Deza commit 81e35b941659a0f6dff8a935c27c7dd6d5cc4213 Author: Sage Weil Date: Wed May 24 09:48:11 2017 -0400 qa/suites/rados/singleton-nomsgr/*: set sortbitwise after upgrade Signed-off-by: Sage Weil commit f2814e4dbb2821a415310559fce405c2eab23947 Author: huanwen ren Date: Tue Dec 27 10:54:45 2016 +0000 mon/OSDMonitor: fixup sortbitwise flag warning "ceph -s" does not report warning when using command "ceph osd unset sortbitwise" to drop sortbitwise flag. we should use "osdmap.get_up_osd_features() & CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT" instead of "(osdmap.get_features(CEPH_ENTITY_TYPE_OSD, NULL) & CEPH_FEATURE_OSD_BITWISE_HOBJ_SORT)", because osdmap.get_features only get local "features" Signed-off-by: huanwen ren (cherry picked from commit c25ee187e28724846d0011cd8145e16956d3636e) commit 02617188688eebde759c375a2257e076e4538491 Author: Sage Weil Date: Tue Nov 1 12:09:57 2016 -0400 mon: remove config option to disable no sortbitwise warning We'll require this soon. Signed-off-by: Sage Weil (cherry picked from commit 04e3319e8aecde9ca58ccb7c89016f9079c7d657) commit 966f2229178c5d72722982d0f6a40f4d01210d9a Merge: 6537fc741c 4ceaa7cce9 Author: Nathan Cutler Date: Wed May 17 09:29:21 2017 +0200 Merge pull request #13450 from dreamhost/wip-18887-jewel jewel: msg: IPv6 Heartbeat packets are not marked with DSCP QoS - simple messenger Reviewed-by: Josh Durgin commit bb79663490468b4ac2832aa9e3cbac019c1f712d Author: Jan Fajerski Date: Thu Apr 20 18:38:43 2017 +0200 fs: normalize file open flags internally used by cephfs The file open flags (O_foo) are platform specific. Normalize these flags before they are send to the MDS. For processing of client messages the MDS should only compare to these normalized flags. Otherwise this can lead to bogus flags getting transmitted on ppc64. Signed-off-by: Jan Fajerski (cherry picked from commit 88d2da5e93198e69435e288ce00d216d5fe27f80) Conflicts: src/client/Client.cc Conflicts can be resolved by choosing changes from HEAD and adding a call to ceph_flags_sys2wire where flags are logged. src/mds/Server.cc Conflicts can be resolved by choosing changes from HEAD and while making sure that the MDS compares request O_ flags the the CEPH_O_ flags, since all wire O_ flags are normalized. commit 4ceaa7cce9c9132d47564d79204b48b1d02e531c Author: Robin H. Johnson Date: Wed May 3 22:31:40 2017 -0700 msg/simple/Pipe: manual backport of fix in PR#14795 Manual backport of errno fixup from PR#14795 (6f1037e22c2a304795895498cdc955e0ef80f8e3), as noted by https://github.com/ceph/ceph/pull/13450#discussion_r114696885. Signed-off-by: Robin H. Johnson commit 6537fc741c3267472e607e581c002854d8eafd46 Merge: af31b453f4 82ea0971b3 Author: Nathan Cutler Date: Wed May 3 13:04:27 2017 +0200 Merge pull request #14667 from smithfarm/wip-18699-jewel jewel: client: fix the cross-quota rename boundary check conditions Reviewed-by: Gregory Farnum commit af31b453f404cb73ee7bdb8b6b02d71ad8aab7e7 Merge: 472034cbe0 335258f975 Author: Nathan Cutler Date: Wed May 3 10:33:24 2017 +0200 Merge pull request #14332 from shinobu-x/wip-19396-jewel jewel: Objecter::epoch_barrier isn't respected in _op_submit() Reviewed-by: Josh Durgin commit 472034cbe0735131f3419205e9f7e6530377bfba Merge: afe98ba074 acf608a903 Author: Nathan Cutler Date: Wed May 3 10:32:37 2017 +0200 Merge pull request #14204 from dzafman/wip-18533-jewel jewel: core: two instances of omap_digest mismatch Reviewed-by: Josh Durgin commit afe98ba07436ea105f4c9c42a52795ce3757419e Merge: 0353a91bfd 043d70461c Author: Nathan Cutler Date: Wed May 3 10:31:38 2017 +0200 Merge pull request #13884 from shinobu-x/wip-19119-jewel jewel: pre-jewel "osd rm" incrementals are misinterpreted Reviewed-by: Josh Durgin commit 0353a91bfd799077ddd87de901e8a07f5da6a13d Merge: 460b12c259 608785a007 Author: Nathan Cutler Date: Wed May 3 10:30:32 2017 +0200 Merge pull request #13647 from mslovy/wip-19083-jewel jewel: osd: preserve allocation hint attribute during recovery Reviewed-by: Gregory Farnum commit 460b12c259f5563d9d1b2477149fe79486ba5bcd Merge: 630cfca36c 905c4acb99 Author: Nathan Cutler Date: Fri Apr 28 10:24:35 2017 +0200 Merge pull request #14791 from smithfarm/wip-19774-jewel jewel: osd: promote throttle parameters are reversed Reviewed-by: Sage Weil commit 630cfca36c5a42a58882966c5598752aac9ff54e Merge: 013e781b45 3ec1a9bf16 Author: Nathan Cutler Date: Fri Apr 28 10:23:45 2017 +0200 Merge pull request #14763 from smithfarm/wip-19562-jewel jewel: api_misc: [ FAILED ] LibRadosMiscConnectFailure.ConnectFailure Reviewed-by: Sage Weil commit 99c65bbc1899663d0b23e2cec3d6e516d0e95ad7 Author: Yehuda Sadeh Date: Mon Jan 9 13:04:43 2017 -0800 rgw: new rest api to retrieve object layout Signed-off-by: Yehuda Sadeh (cherry picked from commit 2768583dc486109e49d209243675b99fdd39e92c) commit 33745a342433716ad8a07ef846dbea9b57de5017 Author: Yehuda Sadeh Date: Thu Jan 5 13:47:24 2017 -0800 rgw: rest api to read zone config params Signed-off-by: Yehuda Sadeh (cherry picked from commit a220a2efbfd675e6abf14ae33c21005bcbf6dadf) commit 013e781b45e28af15da4ddc32025aad713f128e8 Merge: d144d99f00 b698d1fa4c Author: Nathan Cutler Date: Thu Apr 27 21:46:41 2017 +0200 Merge pull request #14481 from dillaman/wip-19468-jewel jewel: librbd: is_exclusive_lock_owner API should ping OSD Reviewed-by: Jason Dillaman commit d144d99f008776dc02c5838feb5a9eb9b84a400b Merge: 0f0cd82eda dbe90c79b8 Author: Nathan Cutler Date: Thu Apr 27 21:05:11 2017 +0200 Merge pull request #14666 from smithfarm/wip-19612-jewel jewel: librbd: Issues with C API image metadata retrieval functions Reviewed-by: Jason Dillaman commit 0f0cd82edabcb2feb0b29793e5b45eb424f1c42d Merge: eba821ce9c 216156b5d4 Author: Nathan Cutler Date: Thu Apr 27 21:04:27 2017 +0200 Merge pull request #14664 from smithfarm/wip-19325-jewel jewel: rbd: [api] temporarily restrict (rbd_)mirror_peer_add from adding multiple peers Reviewed-by: Jason Dillaman commit eba821ce9c61bcb197895e6949ce30a5c07097fa Merge: 25e29c9215 b8fd297eb6 Author: Nathan Cutler Date: Thu Apr 27 20:46:08 2017 +0200 Merge pull request #14694 from ceph/wip-bp-systemd [backport] qa/tasks: systemd test backport to jewel Reviewed-by: Nathan Cutler commit 25e29c921557883da4bf48ff2496bcec6448e6ec Merge: d7b63e21ca 7c6c3c753c Author: Nathan Cutler Date: Thu Apr 27 08:44:26 2017 +0200 Merge pull request #13544 from shinobu-x/wip-18932-jewel jewel: tests: 'ceph auth import -i' overwrites caps, should alert user before overwrite Reviewed-by: Kefu Chai commit acf608a9034e915e38ccea6002ee808c46620433 Author: David Zafman Date: Mon Mar 20 17:28:45 2017 -0700 filestore, tools: Fix logging of DBObjectMap check() repairs Signed-off-by: David Zafman (cherry picked from commit 1704f62c0831e6b07138f7dd14a89fef3c9ed2c1) commit fecc52338b2a58bf1730f7b7a3e4a293e45160d2 Author: David Zafman Date: Fri Mar 3 15:04:02 2017 -0800 osd: Simplify DBObjectMap by no longer creating complete tables Bump the version for new maps to 3 Make clone less efficient but simpler Add rename operation (use instead of clone/unlink) For now keep code that understands version 2 maps Signed-off-by: David Zafman (cherry picked from commit 738156a99ed1caf61e5a8230eb8048360056c08e) No ghobject_t::operator>() so use Kraken cmp_bitwise() instead Need to use MIN_GHOBJ/MAX_GHOBJ instead of std::min/std::max commit 6902c3141eeaefaacd92f33877cf319872f626c7 Author: David Zafman Date: Wed Feb 15 16:17:32 2017 -0800 ceph-osdomap-tool: Fix seg fault with large amount of check error output Signed-off-by: David Zafman (cherry picked from commit 1dda0411f4fbb14ce1e0062da9f14ec3af505d39) commit 4a3e4bcf40cd004fc53e7be467a29084dedc3e1c Author: David Zafman Date: Wed Feb 15 15:02:33 2017 -0800 osd: Add automatic repair for DBObjectMap bug Add repair command to ceph-osdomap-tool too Under some situations the previous rm_keys() code would generated a corrupt complete table. There is no way to figure out what the table should look like now. By removing the entries we fix the corruption and aren't much worse off because the corruption caused some deleted keys to re-appear. This doesn't breaking the parent/child relationship during repair because some of the keys may still be contained in the parent. Signed-off-by: David Zafman (cherry picked from commit 4cd3c74c928a32e065ed9543d6c91d8718a6ae3d) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit d4f0ac0a405266f638b25ec475c0110741e3c431 Author: David Zafman Date: Wed Feb 15 14:59:40 2017 -0800 ceph-osdomap-tool: Fix tool exit status Signed-off-by: David Zafman (cherry picked from commit 666f14ed90655a2d1bedde8561949625db7a9e6c) commit 5f36c319cd7fa4be15efd27b8aabbebc99d8999c Author: Samuel Just Date: Fri Feb 10 15:51:42 2017 -0800 DBObjectMap: rewrite rm_keys and merge_new_complete Leverage the updated in_complete_region and needs_parent to simplify these methods. Signed-off-by: Samuel Just (cherry picked from commit c4dffb68eaafe724f7fdae93a4285a7f8003ea29) commit 1fe4b856a37b29c85a9317e514c7f15f8e4905d5 Author: Samuel Just Date: Fri Feb 10 15:50:57 2017 -0800 DBObjectMap: strengthen in_complete_region post condition Previously, in_complete_region didn't guarantee anything about where it left complete_iter pointing. It will be handy for complete_iter to be pointing at the lowest interval which ends after to_test. Make it so. Signed-off-by: Samuel Just (cherry picked from commit 97b35f4d7d4862da4b6f50ecaef0d292a671fd04) commit 85f2151fec991e5db13d8e6f44b27e092605fb35 Author: Samuel Just Date: Fri Feb 10 15:48:57 2017 -0800 DBObjectMap: fix next_parent() The previous implementation assumed that lower_bound(parent_iter->key()) always leaves the iterator on_parent(). There isn't any guarantee, however, that that key isn't present on the child as well. Signed-off-by: Samuel Just (cherry picked from commit 74a7631d0938d7b44894f022224eab10a90d5cec) commit 484ccda5c53a8ee151b9f97687edd160ad7ebbd7 Author: Samuel Just Date: Thu Feb 9 10:47:59 2017 -0800 test_object_map: add tests to trigger some bugs related to 18533 Signed-off-by: Samuel Just (cherry picked from commit f131dbcf5bb17107c029f942a57e9bf4432a26ee) commit cdeb690869bf6dd52226476c21514cf03ff37d6e Author: David Zafman Date: Tue Feb 14 12:40:33 2017 -0800 test: Add ceph_test_object_map to make check tests Signed-off-by: David Zafman (cherry picked from commit 0e97a01bd7291458881ee53cece2d887f6333669) commit cf5d588d39498c8d65bc64bd2935aecac8546e40 Author: David Zafman Date: Wed Feb 8 18:56:27 2017 -0800 ceph-osdomap-tool: Add --debug and only show internal logging if enabled Signed-off-by: David Zafman (cherry picked from commit 5fb2b2d13953979e5da9f571ab8c4b0b510b8368) commit 4c4a06ff525d9fa2271099db73701c7994054d36 Author: David Zafman Date: Wed Feb 8 18:55:48 2017 -0800 osd: DBOjectMap::check: Dump complete mapping when inconsistency found Signed-off-by: David Zafman (cherry picked from commit fcf1e17c645e8fad5216c3e59627c817e5c858c7) commit 6c128ff8c6944e57059008959f49bd03635c5417 Author: David Zafman Date: Wed Feb 8 15:38:51 2017 -0800 test_object_map: Use ASSERT_EQ() for check() so failure doesn't stop testing Signed-off-by: David Zafman (cherry picked from commit 053a273cbc02d6902a4bb1f11db1ea946498df3a) commit aa769a9d08ac7490f5063bb44c3a44c77a5d7232 Author: David Zafman Date: Wed Feb 8 10:02:40 2017 -0800 tools: Check for overlaps in internal "complete" table for DBObjectMap Changed check to return an error count and fix tool error message Signed-off-by: David Zafman (cherry picked from commit e5e8eb962db6187ea19b96ba29ac83469c90b4ea) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit 761ee7c6af8802ab6b668a0b7ccaa819b2764456 Author: David Zafman Date: Wed Feb 8 09:40:49 2017 -0800 tools: Add dump-headers command to ceph-osdomap-tool Signed-off-by: David Zafman (cherry picked from commit f4101591ad701a62fe027c4744ca8ea505f44bdc) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit 117db1c6d6952d181614f4fe22f0b1866eed6f10 Author: David Zafman Date: Mon Feb 6 21:09:42 2017 -0800 tools: Add --oid option to ceph-osdomap-tool Signed-off-by: David Zafman (cherry picked from commit 2d94889e9ee3359017b1efd560f3557ce03ccee6) commit 4d8120d3b3975ce101f6272240e5e43bcfe0e742 Author: David Zafman Date: Mon Feb 6 21:31:18 2017 -0800 osd: Remove unnecessary assert and assignment in DBObjectMap Fix and add comment(s) Signed-off-by: David Zafman (cherry picked from commit 937e6a03ea4692cc44d53faa0615f8e808c9eb03) commit 86980a045b7176428062a19758d145e38e515b40 Author: snakeAngel2015 Date: Mon Jul 18 14:51:37 2016 +0800 rgw: add suport for creating S3 type subuser of admin rest api Fixes: http://tracker.ceph.com/issues/16682 The original code cannot support create s3 type subuser of admin rest api as when i execute the following command: ./s3curl.pl --id=personal --put -- http://radosgw.h3c.com:8000/admin/user?subuser\&uid=yrf2\&subuser=yrf2:yrf1\&key-type=s3 -v it would return msg as follows : < HTTP/1.1 403 Forbidden < Date: Thu, 14 Jul 2016 07:04:40 GMT * Server Apache/2.4.7 (Ubuntu) is not blacklisted < Server: Apache/2.4.7 (Ubuntu) < x-amz-request-id: tx00000000000000006608f-0057873988-8551-slave < Accept-Ranges: bytes < Content-Length: 114 < Content-Type: application/json < * Connection #0 to host slave.com left intact {"Code":"InvalidAccessKeyId","RequestId":"tx00000000000000006608f-0057873988-8551-slave","HostId":"8551-slave-us"} But i have modified the codes for support it ,and it will return actual msg as follows : "subusers": [ { "id": "yrf2:yrf1", "permissions": "" } ], "keys": [ { "user": "yrf2", "access_key": "B46PXYFEWUX0IMHGKP8C", "secret_key": "2JYxywXizqwiiMd74UXrJdSJMPNlBtYwF5z8rNvh" }, { "user": "yrf2:yrf1", "access_key": "INO55WXJ7JQ1ZZGSAB6B", "secret_key": "GgCKEfF9hArV2hglunbO7KtvKZnbhmsDpqjSj5DL" } ], Please check it ,thanks . Signed-off-by: snakeAngel2015 (cherry picked from commit 6535f6ad2137ee55bf5531e865c05aa10bd39bd0) commit d7b63e21ca9818e21afeb2945b882e24b7a9b10b Merge: a3fae531cd f32b5c613a Author: Nathan Cutler Date: Wed Apr 26 21:32:31 2017 +0200 Merge pull request #14809 from tchaikov/wip-18193-jewel jewel: tests: test/librados/tmap_migrate: g_ceph_context->put() upon return Reviewed-by: Nathan Cutler commit a3fae531cdb8c2c64c66d38c947ca139f4ede83d Merge: 89d6ddb1f5 6c4826606d Author: Nathan Cutler Date: Wed Apr 26 21:32:09 2017 +0200 Merge pull request #14701 from smithfarm/wip-18193-jewel jewel: core: transient jerasure unit test failures Reviewed-by: Kefu Chai commit 89d6ddb1f5ee08b0f019dfaadc5857a788c92a09 Merge: c5f1fce21d be9e83281b Author: Matt Benjamin Date: Wed Apr 26 14:28:29 2017 -0400 Merge pull request #14776 from linuxbox2/jewel-pullup-civet-chunked [DNM] jewel: pullup civet chunked commit c5f1fce21d0996cce751b6bcca5e57da2cafc135 Merge: 4d97e0ba8e 97cd21afc5 Author: Nathan Cutler Date: Wed Apr 26 19:32:07 2017 +0200 Merge pull request #14416 from smithfarm/wip-19557-jewel jewel: tests: upgrade/hammer-x failing with OSD has the store locked when Thrasher runs ceph-objectstore-tool on down PG Reviewed-by: Kefu Chai commit be9e83281b8c765cd111d5687a516fcd3ca521a5 Author: Matt Benjamin Date: Tue Apr 25 09:44:39 2017 -0400 civetweb: pullup chunked encoding by Marcus Fixes: http://tracker.ceph.com/issues/19736 Signed-off-by: Matt Benjamin commit 608785a0079f807ff860c56d96b1b67bf6a2ed74 Author: yaoning Date: Fri Jun 24 09:51:07 2016 +0800 os: make zero values noops for set_alloc_hint() in FileStore Signed-off-by: yaoning (cherry picked from commit e2ec24f61b55457caccefecd56f9f08b98264802) commit f32b5c613a8acd50e32747c3581131d28d209efa Author: Kefu Chai Date: Wed Apr 26 22:58:30 2017 +0800 test/librados/tmap_migrate: g_ceph_context->put() upon return Signed-off-by: Kefu Chai Conflict: test/librados/tmap_migrate.cc this change is not cherry-picked from master, because tmap_migrate was removed in master. so we are applying the same change in cb1cda96713b2ec0f6418c4cbe3d964c2020729c to this test. commit 905c4acb99f9ea78ff615034dae969ab089bda06 Author: Mark Nelson Date: Tue May 3 09:56:47 2016 -0500 Fix reveresed promote throttle default parameters. Signed-off-by: Mark Nelson (cherry picked from commit 793ceac2f3d5a2c404ac50569c44a21de6001b62) commit fb3ee2efcc13c37db90faa1bc8bad584bab22efa Author: Marcus Watts Date: Thu Apr 13 05:33:55 2017 -0400 rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0 Keystone tokens can be revoked. This causes them to fail validation. However, in ceph, we cache them. As long as they're in the cache we trust them. To find revoked tokens there's a call OSI-PKI/revoked but that's only useful for pki tokens. Installations using fernet/uuid may not even have the proper credentials to support the call, in which case the call blows up in various ways filling up logs with complaints. This code makes the revocation thread optional; by disabling it, the complaints go away. A further fix is in the works to use other more modern calls available in modern keystone installations to properly deal with non-PKI/PKIZ tokens. (NB: jewel has this logic in src/rgw/rgw_swift.cc not in src/rgw/rgw_keystone.h) To disable the revocation thread, use at least one of these: rgw_keystone_token_cache_size = 0 using this will cause tokens to be validated on every call. You may instead want to set rgw_keystone_revocation_interval = 0 using just this will disable the revocation thread, but leaves the cache in use. That avoids the extra validation overhead, but means token revocation won't work very well. Fixes: http://tracker.ceph.com/issues/9493 Fixes: http://tracker.ceph.com/issues/19499 Signed-off-by: Marcus Watts (cherry picked from commit 003291a8cbca455c0e8731f66759395a0bb1f555) commit 4d97e0ba8ebc89b7797b0936a9e046ef59cc3899 Merge: cc820a0d4c c2efeb4b62 Author: Nathan Cutler Date: Tue Apr 25 21:06:26 2017 +0200 Merge pull request #14686 from smithfarm/wip-19686-jewel jewel: osd: Give requested scrubs a higher priority Reviewed-by: David Zafman commit e552d91f73d996c44821f5fbfb28cfc3e5cddc9a Author: Matt Benjamin Date: Sat Nov 5 13:13:47 2016 -0400 rgw: add bucket size limit check to radosgw-admin The change adds a new list of all buckets x all users, with fields for bucket name, tenant name, current num_objects, current num_shards, current objects per shard, and the corresponding fill_status--the latter consisting of 'OK', 'WARN %', or 'OVER %.' The warning check is relative to two new tunables. The threshold max objects per shard is set as rgw_bucket_safe_max_objects_per_shard, which defaults to 100K. The value rgw_bucket_warning_threshold is a percent of the current safe max at which to warn (defaults to 90% of full). From review: * fix indentation (rgw_admin) * if user a user_id is provided, check only buckets for that user * update shard warn pct to be pct-of-fill (not 100 - pct-of-fill) * print only buckets near or over per-shard limit, if --warnings-only * s/bucket limitcheck/bucket limit check */ * sanity shard limit should be 90, not 10 (because that changed) * fixes for memleaks and other points found by cbodley Fixes: http://tracker.ceph.com/issues/17925 Signed-off-by: Matt Benjamin (cherry picked from commit 7bc144ce36fedc16a3dedc54598b0d75fb8c68bc) commit cc820a0d4c2676799383aeaf49a9269e104853ce Merge: 5ee54cfe8f d079b91479 Author: Nathan Cutler Date: Tue Apr 25 17:38:21 2017 +0200 Merge pull request #14605 from asheplyakov/19476-jewel jewel: rgw: don't return skew time in pre-signed url Reviewed-by: Casey Bodley commit 5ee54cfe8f0a955b11edb4819a652ee420f43e5d Merge: 37254aee6c c05bd1cb1f Author: Nathan Cutler Date: Tue Apr 25 17:37:16 2017 +0200 Merge pull request #14660 from smithfarm/wip-19478-jewel jewel: rgw: zonegroupmap set does not work Reviewed-by: Casey Bodley commit 37254aee6c3f725cf60deb5c7a3fc28405c51216 Merge: 09919f938f faeb8088ac Author: Nathan Cutler Date: Tue Apr 25 17:36:23 2017 +0200 Merge pull request #14607 from asheplyakov/19607-jewel jewel: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote Reviewed-by: Casey Bodley commit 09919f938f3c1e9e81963f676a9dd13114d54e73 Merge: d5e1345a2a 527911fab7 Author: Nathan Cutler Date: Tue Apr 25 15:50:26 2017 +0200 Merge pull request #14587 from asheplyakov/19617-jewel jewel: mon/MonClient: make get_mon_log_message() atomic Reviewed-by: Sage Weil commit 6c4826606dc81fdd847959a49454c69a958bb1d8 Author: Kefu Chai Date: Mon Dec 5 20:23:21 2016 +0800 test/ceph_crypto: do not read ceph.conf in global_init() ForkDeathTest.MD5 expect an empty output while global_init() complains if ceph.conf is missing if 0 is passed in as the `flags`. this test passes if ceph.conf is in current working directory, but jenkins does not prepare this file for this test. Fixes: http://tracker.ceph.com/issues/18128 Signed-off-by: Kefu Chai (cherry picked from commit c72a2271a8012a66d7bbccf5766a73da5bb878d6) Conflicts: src/test/ceph_crypto.cc - jewel does not have 5af29540675b674c1985ff98b28a783ed124acf6 commit aca2659952528aa1b40b02828293342128657194 Author: Loic Dachary Date: Thu Dec 8 12:40:42 2016 +0100 tests: fix erasure-code premature deallocation of cct The setup function returns before the run function, the cct variable must be a data member, not a local variable that gets de-allocated before run() starts. Signed-off-by: Loic Dachary (cherry picked from commit efa1e54362423d4cfd1541fb8c68237b7b9ebbe3) commit 8bddd427d9a4bca75a352ca333847dbd6d2c369c Author: Pan Liu Date: Thu Feb 16 22:17:52 2017 +0800 rbd-nbd: no need create asok file for unmap and list-mapped commands. Fixes: http://tracker.ceph.com/issues/17951 Signed-off-by: Pan Liu (cherry picked from commit 72352653d585ef89043a4ece371b5c0cb3f6f32a) commit 328bfbd25c63b1b0e253865abedada7a9e5858e4 Author: Jason Dillaman Date: Tue Nov 29 12:36:00 2016 -0500 rbd-nbd: restart parent process logger after forking Fixes: http://tracker.ceph.com/issues/18070 Signed-off-by: Jason Dillaman (cherry picked from commit 29baf254d72cc593572b5a6215360ba51e3be198) commit 192e7bcdcd1bc02b40b29eff86a335cc8919f663 Author: Kefu Chai Date: Wed Nov 23 19:45:57 2016 +0800 crushtool: do not release g_ceph_context at exit it is but a work around of occasionally timeout. Signed-off-by: Kefu Chai (cherry picked from commit d305cc51b18cbf4b2757bbacb5d43324461306a9) commit 8a2f27cc632c26d7c2b8e8528b4d459b1d78705b Author: Kefu Chai Date: Tue Nov 15 14:21:03 2016 +0800 common,test: g_ceph_context->put() upon return prior to this change, global_init() could create a new CephContext and assign it to g_ceph_context. it's our responsibilty to release the CephContext explicitly using cct->put() before the application quits. but sometimes, we fail to do so. in this change, global_init() will return an intrusive_ptr, which calls `g_ceph_context->put()` in its dtor. this ensures that the CephContext is always destroyed before main() returns. so the log is flushed before _log_exp_length is destroyed. there are two cases where global_pre_init() is called directly. - ceph_conf.cc: g_ceph_context->put() will be called by an intrusive_ptr<> deleter. - rgw_main.cc: global_init() is called later on on the success code path, so it will be taken care of. Fixes: http://tracker.ceph.com/issues/17762 Signed-off-by: Kefu Chai (cherry picked from commit cb1cda96713b2ec0f6418c4cbe3d964c2020729c) Conflicts: src/ceph_fuse.cc src/ceph_mgr.cc src/global/global_init.cc src/rgw/rgw_main.cc src/test/compressor/test_compression.cc src/test/compressor/test_compression_plugin.cc src/test/compressor/test_compression_plugin_snappy.cc src/test/compressor/test_compression_plugin_zlib.cc src/test/compressor/test_compression_snappy.cc src/test/compressor/test_compression_zlib.cc src/test/erasure-code/TestErasureCode.cc src/test/erasure-code/TestErasureCodeExample.cc src/test/erasure-code/TestErasureCodeIsa.cc src/test/erasure-code/TestErasureCodeJerasure.cc src/test/erasure-code/TestErasureCodeLrc.cc src/test/erasure-code/TestErasureCodePlugin.cc src/test/erasure-code/TestErasureCodePluginIsa.cc src/test/erasure-code/TestErasureCodePluginJerasure.cc src/test/erasure-code/TestErasureCodePluginLrc.cc src/test/erasure-code/TestErasureCodePluginShec.cc src/test/erasure-code/TestErasureCodeShec.cc src/test/erasure-code/TestErasureCodeShec_thread.cc src/test/fio/fio_ceph_objectstore.cc src/test/librados/misc.cc src/test/mon/PGMap.cc src/test/msgr/test_async_networkstack.cc src/test/msgr/test_userspace_event.cc src/test/objectstore/Allocator_test.cc src/test/objectstore/BitAllocator_test.cc src/test/objectstore/test_bluefs.cc src/test/objectstore/test_bluestore_types.cc src/test/objectstore/test_memstore_clone.cc src/test/osd/TestPGLog.cc src/test/rgw/test_http_manager.cc src/test/rgw/test_rgw_compression.cc src/test/test_mempool.cc src/tools/rados/rados.cc commit d5e1345a2a6bd8456417db6bb60d61871165e6bd Merge: 256f48f463 5096fc9c7c Author: Nathan Cutler Date: Tue Apr 25 15:42:43 2017 +0200 Merge pull request #14665 from smithfarm/wip-19610-jewel jewel: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value Reviewed-by: Jason Dillaman commit 256f48f46354f11a2f238f24fa9d890b55f4f4fc Merge: 4f67da1b9d 21a83e1276 Author: Nathan Cutler Date: Tue Apr 25 09:20:19 2017 +0200 Merge pull request #14653 from smithfarm/wip-19662-jewel jewel: rgw_file: fix event expire check, don't expire directories being read Reviewed-by: Matt Benjamin commit 4f67da1b9d554a1a5371665573b1f56d3f903de3 Merge: c2452c5364 33af18e592 Author: Loic Dachary Date: Tue Apr 25 09:17:33 2017 +0200 Merge pull request #14635 from smithfarm/wip-19690-jewel jewel: doc: Improvements to crushtool manpage Reviewed-by: Loic Dachary commit 040ff013d5c8aded8beedc59bbc4f5afccc64e46 Author: Kefu Chai Date: Wed Nov 16 11:56:09 2016 +0800 crushtool: s/exit(EXIT_FAILURE)/return EXIT_FAILURE/ so the destructor(s) can be called. Signed-off-by: Kefu Chai (cherry picked from commit e01b89ed6be6b99fec5c725f4bc5769b42468cac) Conflicts: src/tools/crushtool.cc - jewel does not have 17feefbcb3105553b763cb7ce123b20b77b95857 commit 8e993e6e43ecd6e2b444961d1e6c945081b64207 Author: Kefu Chai Date: Wed Nov 16 11:19:04 2016 +0800 global/signal_handler: reset injected segv after test ~CephContext() => ~TypedSingletonWrapper() => ~MempoolObs => unregister_command() => ldout() << "unregister_command" => Log::submit_entry() => *(volatile int *)(0) = 0xdead; Signed-off-by: Kefu Chai (cherry picked from commit d932c8f2f23263924103a900714db82ee87f6eef) Conflicts: src/log/Log.cc - jewel has "namespace log" instead of "namespace logging" (trivial resolution) commit f1c0042b831826e5bd72ada79a4918a9c26bda24 Author: Kefu Chai Date: Tue Nov 15 14:42:35 2016 +0800 test_cors.cc: fix the mem leak Signed-off-by: Kefu Chai (cherry picked from commit c8a3777203482cabf6739a8ba69b127df8697628) commit c49b114e8dde6c3f3c5a5b663bd8299b4f2feafb Author: weiqiaomiao Date: Tue Sep 6 16:34:52 2016 +0800 rgw: fix failed to create bucket if a non-master zonegroup has a single zone If a non-master zonegroup has a single zone, the metadata sync thread not running and the non-master zonegroup can't sync user from master zonegroup, so we can't create bucket(or other metadata update) in it because the authenticated user not found in the zone of non-master zonegroup. Signed-off-by: weiqiaomiao (cherry picked from commit 949af79b21098e6410bc29274cf36eae2d89faea) Conflicts: src/rgw/rgw_rados.cc - retain d32654b7cd60ccc4e23d3f05b9e4385a697bacd6 which was merged after this commit commit 43327f83efcbc5ce54be866a30da8dd1e58d6707 Author: Loic Dachary Date: Wed Feb 22 01:49:12 2017 +0100 ceph-disk: dmcrypt activate must use the same cluster as prepare When dmcrypt is used, the fsid cannot be retrieved from the data partition because it is encrypted. Store the fsid in the lockbox to enable dmcrypt activation using the same logic as regular activation. The fsid is used to retrive the cluster name that was used during prepare, reason why activation does not and must not have a --cluster argument. Fixes: http://tracker.ceph.com/issues/17821 Signed-off-by: Loic Dachary (cherry picked from commit 7f66672b675abbc0262769d32a38112c781fefac) Conflicts: src/ceph-disk/ceph_disk/main.py - in master, self.create_key() takes an argument (self.args.cluster) but in jewel it takes no argument commit c2452c53641fca416268c2e31c774b8b7e609c88 Merge: 82b8c89e47 01d04e28db Author: Nathan Cutler Date: Tue Apr 25 07:20:41 2017 +0200 Merge pull request #13608 from smithfarm/wip-19063-jewel jewel: tests: eliminate race condition in Thrasher constructor Reviewed-by: Kefu Chai Reviewed-by: David Zafman commit 3ec1a9bf16e2c305096e11223aaa8db94dc4084d Author: Sage Weil Date: Fri Mar 31 10:06:42 2017 -0400 ceph_test_librados_api_misc: fix stupid LibRadosMiscConnectFailure.ConnectFailure test Sometimes the cond doesn't time out and it wakes up instead. Just repeat the test many times to ensure that at least once it times out (usually it doesn't; it's pretty infrequent that it doesn't). Fixes: http://tracker.ceph.com/issues/15368 Signed-off-by: Sage Weil (cherry picked from commit 8bc197400d94ee2716d3f2fa454247379a676cf9) commit 82b8c89e477610641a6e21123b7c929e9c560729 Merge: 28c7ce595a 30c9527353 Author: Nathan Cutler Date: Mon Apr 24 22:27:33 2017 +0200 Merge pull request #14195 from cbodley/wip-19353 jewel: rgw: use separate http_manager for read_sync_status Reviewed-by: Yehuda Sadeh commit 28c7ce595ab8fef7c0076f091f249f6b3548ad54 Merge: c05ecff3f0 2e50fe1684 Author: Nathan Cutler Date: Mon Apr 24 22:26:50 2017 +0200 Merge pull request #14066 from asheplyakov/19321-bp-jewel jewel: rgw: fix break inside of yield in RGWFetchAllMetaCR Reviewed-by: Yehuda Sadeh commit c05ecff3f0cb5af2f3aa52b68188742796335ddb Merge: 3240cbf4ef dc4e7a1a86 Author: Nathan Cutler Date: Mon Apr 24 22:26:14 2017 +0200 Merge pull request #14064 from asheplyakov/19211-bp-jewel jewel: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log Reviewed-by: Yehuda Sadeh commit 3240cbf4ef60733f55800039051e85f2a7ace61d Merge: d4672acbb8 85fbb00f6e Author: Nathan Cutler Date: Mon Apr 24 22:25:21 2017 +0200 Merge pull request #13842 from smithfarm/wip-19145-jewel jewel: rgw: a few cases where rgw_obj is incorrectly initialized Reviewed-by: Yehuda Sadeh commit d4672acbb82daf81eaf259e659dd627ec9f3bc79 Merge: 61ed719d75 ec0668c201 Author: Nathan Cutler Date: Mon Apr 24 22:24:29 2017 +0200 Merge pull request #13837 from smithfarm/wip-19048-jewel jewel: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync() Reviewed-by: Yehuda Sadeh commit 61ed719d75bf3606fa9e3ae8c8ed776aa2ee4313 Merge: f5e51db564 ced799f9c6 Author: Nathan Cutler Date: Mon Apr 24 22:23:24 2017 +0200 Merge pull request #13724 from asheplyakov/18626-bp-jewel jewel: rgw: Use decoded URI when verifying TempURL Reviewed-by: Yehuda Sadeh commit 66c3db7aee5b53f83e87ee8c8f081ab9b0336177 Author: Casey Bodley Date: Wed Mar 8 16:31:34 2017 -0500 rgw: data sync skips slo data when syncing the manifest object Fixes: http://tracker.ceph.com/issues/19027 Signed-off-by: Casey Bodley (cherry picked from commit 8b69847d7b3e92c70090d1dddf7cea5c44fb6b20) Conflicts: bucket cleanup, overrides commit 303a62f7f15c69413165ed604869909587714a94 Author: Casey Bodley Date: Mon Mar 13 11:33:02 2017 -0400 rgw: RGWGetObj applies skip_manifest flag to SLO Signed-off-by: Casey Bodley (cherry picked from commit 987377ae34382e107e1d54f0bfc1121fcedb4513) commit f3d99ae8aafaa65e91cd233b5ce8054678d8fa11 Author: Casey Bodley Date: Tue Feb 21 10:27:13 2017 -0500 rgw: allow system users to read SLO parts multisite data sync relies on fetching the object as the system user Fixes: http://tracker.ceph.com/issues/19027 Signed-off-by: Casey Bodley (cherry picked from commit d50d18c500fd5dd89e7cada1162cf453b36df370) Conflicts: auth rework commit f5e51db5644d4eafdbe3b2d541582a17febed139 Merge: c90cfb7a32 eac0e27193 Author: Nathan Cutler Date: Fri Apr 21 22:51:59 2017 +0200 Merge pull request #14643 from smithfarm/wip-revert-14427 Wip revert 14427 Reviewed-by: Josh Durgin commit db053da61807e26876d97786550a769295c7955a Author: John Spray Date: Mon Mar 27 12:56:31 2017 +0100 mds: validate prealloc_inos on sessions after load Mitigates http://tracker.ceph.com/issues/16842 Signed-off-by: John Spray (cherry picked from commit c39aaf90ed1b23343eba2b341bb8ee6a50a4ea74) Conflicts: src/mds/InoTable.cc - no 5259683e7819c22c14b21b1dd678a33e14574f21 in jewel src/mds/InoTable.h - no 5259683e7819c22c14b21b1dd678a33e14574f21 in jewel commit 2b5eb8fa141fa8bd9173dee206ec3530d702fc3a Author: John Spray Date: Mon Mar 27 12:33:59 2017 +0100 mds: operator<< for Session Use this to get a nice human readable name when available (also including the session id in parentheses) Signed-off-by: John Spray (cherry picked from commit 0f89787d8312f132ebb621f16c44e950b17a395a) commit 5b562145197833c6a6f48f006dfb13b6e38f9084 Author: Henrik Korkuc Date: Sun Feb 19 11:44:20 2017 +0200 client/Client.cc: add feature to reconnect client after MDS reset Client.cc marks session as stale instead of reconecting after received reset from MDS. On MDS side session is closed so MDS is ignoring cap renew. This adds option to reconnect stale client sessions instead of just marking sessions stale. Fixes: http://tracker.ceph.com/issues/18757 Signed-off-by: Henrik Korkuc (cherry picked from commit e0bbc704676ef4aed510daff075ef63c9e73b7b3) commit 8f21038d30097622e319ab986631b03f87f5d907 Author: Kefu Chai Date: Tue Mar 21 12:49:45 2017 +0800 doc: cephfs: fix the unexpected indent warning Signed-off-by: Kefu Chai (cherry picked from commit e423f0b59711422b40c4b3de0bdc73b0947c04d3) commit f9a1954e113e807b5f9ee6fd56351adc2841730a Author: Barbora Ančincová Date: Thu Feb 16 10:45:36 2017 +0100 doc: additional edits in FUSE client config Signed-off-by: Bara Ancincova (bara@redhat.com) (cherry picked from commit b6cad3364c020abd3acf906643fa0b6cbb862a0a) commit 018649f8a4628881c706a6df9b3eef48403fe91e Author: Barbora Ančincová Date: Thu Jan 26 12:23:34 2017 +0100 doc: Dirty data are not the same as corrupted data Signed-off-by: Bara Ancincova (bara@redhat.com) (cherry picked from commit 80db40f8559128baadad42b925ae813e51a31409) commit 1d8a5b6d64a26a36fb3a227c52908631d95bef79 Author: Barbora Ančincová Date: Mon Jan 23 16:34:55 2017 +0100 doc: minor changes in fuse client config reference Signed-off-by: Bara Ancincova (bara@redhat.com) (cherry picked from commit e57605681f10436f4b2c85e95179a2904b8c80da) commit 1ae46b2b94280b8b4a1db89b2cbed8f12bcbcc53 Author: Patrick Donnelly Date: Sun Jul 24 23:21:29 2016 -0400 doc: add client config ref Fixes: http://tracker.ceph.com/issues/16743 Signed-off-by: Patrick Donnelly (cherry picked from commit 9ad2ccf29830d5309336fc7de877b6926e5dbacd) commit b8fd297eb64cd29c58596d0a5b0be8fbe9d94a72 Author: Vasu Kulkarni Date: Tue Apr 11 13:51:47 2017 -0700 use sudo to check check health Signed-off-by: Vasu Kulkarni (cherry picked from commit 7af157ad4ce7f7e2b8de97ee10eeaf64b9099bc0) commit 1b91ffc0fbe76c5475f17d4e15ea295ee3680688 Author: Vasu Kulkarni Date: Wed Mar 29 09:27:20 2017 -0700 Add reboot case for systemd test test systemd units restart after reboot Signed-off-by: Vasu Kulkarni (cherry picked from commit 7b587304a54d9b21041ffdfbc85fad8d87859c49) commit 3d8d1dad8a698a9fb45be7b31c92f8b71ee67720 Author: Vasu Kulkarni Date: Wed Mar 29 09:56:11 2017 -0700 Fix distro's, point to latest version Signed-off-by: Vasu Kulkarni (cherry picked from commit 1947648669971c1bd1ca189870ed9b25bbd48d3a) commit c90cfb7a327fcbb9508e617c2353becb7e5cb45f Merge: 327276cf3d 9b77b16b88 Author: Nathan Cutler Date: Thu Apr 20 22:05:57 2017 +0200 Merge pull request #14602 from asheplyakov/19646-jewel jewel: ceph-disk: enable directory backed OSD at boot time Reviewed-by: Loic Dachary commit 327276cf3d4fb7a2558620a8a7d9cc90e5d6e5c5 Merge: 7008c64c51 25e43ac256 Author: Nathan Cutler Date: Thu Apr 20 22:04:41 2017 +0200 Merge pull request #14449 from smithfarm/wip-test-doc-oversight tests: fix oversight in yaml comment Reviewed-by: Loic Dachary commit f34489dd52f07aeb88e1dbf361ead63f09bb9c65 Author: YunfeiGuan Date: Mon Apr 10 05:48:47 2017 +0000 cephfs: fix mount point break off problem after mds switch occured The hot-standby become active as we expected but the mount piont broken strangely when the active mds is down. The root reason is the new mds use last_cap_renews decoded from ESesson::replay in find_idle_sessions and wrongly killed the session. Maybe we should reset session->last_cap_renew to the current time when server send OPEN to client in reconnect stage. Fixes: http://tracker.ceph.com/issues/19437 Signed-off-by: Guan yunfei (cherry picked from commit 4ef830c5d6f22bf0d4f82a8624c772ecbbda44a6) Conflicts: src/mds/Server.cc (leave '\n' in because jewel does not have 693132eb00b1803d5e97a79908521d5a6903e9f8; jewel ceph_clock_now takes a CephContext object) commit 7008c64c5143a3ea19d6679a3e521897f74ff69a Merge: d62644abcd f8aa6be06c Author: Nathan Cutler Date: Thu Apr 20 19:26:52 2017 +0200 Merge pull request #13606 from smithfarm/wip-19062-jewel jewel: build/ops: enable build of ceph-resource-agents package on rpm-based os Reviewed-by: Ken Dreyer commit d62644abcde0fd99da24922ce7638d173952ceb4 Merge: 8df8960553 482bd1adab Author: Nathan Cutler Date: Thu Apr 20 18:18:05 2017 +0200 Merge pull request #14654 from smithfarm/wip-19461-jewel jewel: admin ops: fix the quota section Reviewed-by: Casey Bodley commit c2efeb4b62e6dc4e6975561eb8b20cfca66f7237 Author: David Zafman Date: Mon Apr 17 14:58:02 2017 -0700 osd: Give requested scrub work a higher priority Once started we now queue scrub work at higher priority than scheduled scrubs. Fixes: http://tracker.ceph.com/issues/15789 Signed-off-by: David Zafman (cherry picked from commit ebab8b1f4f67fbdec1e147c580329c1e2b5cf7cd) Conflicts: src/osd/OSD.h - in jewel, the PGScrub() call is enclosed within op_wq.queue(make_pair(...)) instead of enqueue_back() commit db86a24e7906ca7c70c60b1752f1230d56361bcd Author: Yan, Zheng Date: Wed Feb 22 17:33:05 2017 +0800 client: wait for lastest osdmap when handling set file/dir layout Fixes: http://tracker.ceph.com/issues/18914 Signed-off-by: "Yan, Zheng" (cherry picked from commit 76f5eb86cdd61dde4e6c7cfeb5cf34f0c0334f21) Conflicts: src/client/Client.cc (jewel does not have 201c56039) src/client/Client.h (jewel does not have 201c56039) commit 7b9283beec0c4bf8e2067558fc4e03e336849b77 Author: Yang Honggang Date: Thu Apr 13 20:09:07 2017 +0800 cephfs: fix write_buf's _len overflow problem After I have set about 400 64KB xattr kv pair to a file, mds is crashed. Every time I try to start mds, it will crash again. The root reason is write_buf._len overflowed when doing Journaler::append_entry(). This patch try to fix this problem through the following changes: 1. limit file/dir's xattr size 2. throttle journal entry append operations Fixes: http://tracker.ceph.com/issues/19033 Signed-off-by: Yang Honggang joseph.yang@xtaotech.com (cherry picked from commit eb915d0eeccbe523f8f70f6571880003ff459459) commit b52c5088618011a569f157616c5c667c2fc1e9fe Author: John Spray Date: Wed Mar 8 12:13:46 2017 +0000 mds: shut down finisher before objecter Some of the finisher contexts would try to call into Objecter. We mostly are protected from this by mds_lock+the stopping flag, but at the Filer level there's no mds_lock, so in the case of file size probing we have a problem. Fixes: http://tracker.ceph.com/issues/19204 Signed-off-by: John Spray (cherry picked from commit 177a97d5c55ee6a2d5dcd3cf0893546190b10f7a) Conflicts: src/mds/MDSRank.cc (no purge_queue.shutdown() in jewel because jewel does not have 8ebf7d95a9071de24bb1e56a6423c505169cb4de) commit 96e801fb53941214c669ac01c1804a4d1f27086c Author: John Spray Date: Tue Mar 28 14:13:33 2017 -0400 mds: ignore ENOENT on writing backtrace We get ENOENT when a pool doesn't exist. This can happen because we don't prevent people deleting former cephfs data pools whose files may not have had their metadata flushed yet. http://tracker.ceph.com/issues/19401 Signed-off-by: John Spray (cherry picked from commit 3fccc2372f2715d075b05e459140360cf6e7ca96) commit 8df896055366d777ca309dd4a7840f2a2a5a72cc Merge: da888fa871 7468689314 Author: Nathan Cutler Date: Thu Apr 20 14:53:30 2017 +0200 Merge pull request #14680 from smithfarm/wip-19711-jewel jewel: [test] test_notify.py: rbd.InvalidArgument: error updating features for image test_notify_clone2 Reviewed-by: Jason Dillaman commit 74686893147f2d7e1fd1287de37587554fea96c6 Author: Jason Dillaman Date: Wed Apr 19 09:26:31 2017 -0400 test: rbd master/slave notify test should test active features Fixes: http://tracker.ceph.com/issues/19692 Signed-off-by: Jason Dillaman (cherry picked from commit 0dcba41cba96566d0b8da54cf0316d523b88ded2) commit 7347f1193963666ce07ff9163064aeb0cf343d38 Author: Yan, Zheng Date: Wed Apr 5 21:29:10 2017 +0800 mds: make C_MDSInternalNoop::complete() delete 'this' Fixes: http://tracker.ceph.com/issues/19501 Signed-off-by: "Yan, Zheng" (cherry picked from commit 424e0c6744e7f63459ca0ff7deab751726aa30cd) Conflicts: src/mds/MDSContext.h (omit "override" because jewel does not have 1a91aeab987870b3ccbcf2f1e476fac8b534d449) commit 824b19a9a66261a90c4e4de64e877a42438febd2 Author: John Spray Date: Wed Mar 29 19:38:37 2017 +0100 tools/cephfs: set dir_layout when injecting inodes When we left this as zero, the MDS would interpret it was HASH_LINUX rather than the default HASH_RJENKINS. Potentially that could cause problems if there perhaps were already dirfrags in the metadata pool that were set up using rjenkins. Mainly it just seems more appropriate to explicitly set this field rather than hit the fallback behaviour. Related: http://tracker.ceph.com/issues/19406 Signed-off-by: John Spray (cherry picked from commit 7d6d542885bd29b71214f9ca52bd26e9183c5d01) commit eab56dae6714706cf9650dff172b20a764f5481c Author: John Spray Date: Thu Mar 9 13:15:46 2017 +0000 mon: fix hiding mdsmonitor informative strings Local `stringstream ss` declarations were hiding the real variable used to feed back to the user. Fixes: http://tracker.ceph.com/issues/16709 Signed-off-by: John Spray (cherry picked from commit 00404ae9bd4cce0518a44d36d2d6a5612f4f9d04) commit da888fa8713178cc23c1069ce2651e02c98aad05 Merge: 8068b546af 2271cd8128 Author: Kefu Chai Date: Thu Apr 20 19:10:33 2017 +0800 Merge pull request #14402 from shinobu-x/wip-17331-jewel jewel: ceph-disk: ceph-disk list reports mount error for OSD having mount options with SELinux context Reviewed-by: Loic Dachary Reviewed-by: Brad Hubbard Reviewed-by: Kefu Chai commit 8068b546afbe8e774c9d699205a32215e38c5d4f Merge: 118ccad0ba 754b4a482c Author: Nathan Cutler Date: Thu Apr 20 13:06:33 2017 +0200 Merge pull request #13865 from smithfarm/wip-19158-jewel jewel: rgw: health check errors out incorrectly Reviewed-by: Radoslaw Zarzynski commit d57437e338984c5db84f9d16387e082ada5a0a33 Author: John Spray Date: Mon Mar 6 11:51:31 2017 +0000 mds: reset heartbeat in export_remaining_imported_caps This loop can be very long. Fixes: http://tracker.ceph.com/issues/19118 Signed-off-by: John Spray (cherry picked from commit 85071f1509beba4a390730e6a3a4332484646d63) commit 6adf1904ed7209b70328b20f248a701fbdd3127c Author: John Spray Date: Mon Mar 6 11:24:50 2017 +0000 mds: heartbeat_reset in dispatch Previously we only heartbeated in tick. However, our locking is not guaranteed to be fair, so on a super-busy dispatch queue it may be possible for the heartbeat to time out while the tick() function is waiting for mds_lock. Fixes: http://tracker.ceph.com/issues/19118 Signed-off-by: John Spray (cherry picked from commit 819394549af10532419d88742fae3a69d2ea487d) commit 63f41d543f8a5f1f55a12612d39c6a2a1cf9c114 Author: Yan, Zheng Date: Wed Feb 15 11:45:26 2017 +0800 test/libcephfs: avoid buffer overflow when testing ceph_getdents() The buffer size should be at least "2 * sizeof(struct dirent)". Otherwise, the code that checks dentry '..' overflow. Fixes: http://tracker.ceph.com/issues/18941 Signed-off-by: "Yan, Zheng" (cherry picked from commit fa6671345b8f3a82dcd232f99e55a982b0a641f1) commit 7146816065bea55b4e3fec59048a459bcff50f55 Author: Zhi Zhang Date: Fri Feb 10 10:56:46 2017 +0800 mds/StrayManager: aviod reusing deleted inode in StrayManager::_purge_stray_logged Signed-off-by: Zhi Zhang (cherry picked from commit 4978e57419482384279d7e784a625f5e5c10961a) commit d8b139b5847cb46aa13486ee85e26f26421f36d6 Author: John Spray Date: Wed Feb 1 00:38:08 2017 +0000 tasks/cephfs: switch open vs. write in test_open_inode Do the write after opening the file, so that we get good behaviour wrt the change in Mount.open_background that uses file existence to confirm that the open happened. Signed-off-by: John Spray (cherry picked from commit a027dba78fc8bc84ae39d7998b386ce21c01e1bf) commit e8ae80fc8721e500782c7e87a7cb89128c2a9687 Author: John Spray Date: Thu Jan 26 16:48:58 2017 +0000 qa: fix race in Mount.open_background Previously a later remote call could end up executing before the remote python program in open_background had actually got as far as opening the file. Fixes: http://tracker.ceph.com/issues/18661 Signed-off-by: John Spray (cherry picked from commit c6d91dd91252e703d08b8ac62ac6a47ee82c0bed) commit 36c86f71efc10f9a651aedf01dc6fb9d1bfed703 Author: Yan, Zheng Date: Wed Jan 25 15:28:23 2017 +0800 mds: don't purge strays when mds is in clientreplay state MDS does not trim log when it's in clientreplay state. If mds hang at clientreplay state (due to bug), purging strays can submit lots of log events and create very large mds log. Signed-off-by: "Yan, Zheng" (cherry picked from commit 86bbc7fff02668077f27d0924ba3efe6544b77f6) commit 8b01cf33575783661a1e1151c8214d327a08b6f9 Author: Yan, Zheng Date: Wed Jan 25 11:03:45 2017 +0800 mds: skip fragment space check for replayed request when handling replayed request, stray directory can be different from the stray directory used by the original request. The fragment space check for stray directory can fail. Fixes: http://tracker.ceph.com/issues/18660 Signed-off-by: "Yan, Zheng" (cherry picked from commit afe889cbc5baab196567c2aad01f49fe90901fda) commit 82ea0971b3cb07c32ec837cb85de63d4068a70d7 Author: Greg Farnum Date: Wed Dec 14 12:09:44 2016 -0800 client: fix the cross-quota rename boundary check conditions We were previously rejecting a rename if either of the involved directories was a quota root, even if the other directory was part of the same quota "tree". What we really want to do is identify the correct quota root (whether local or ancestral) for each directory and compare them. So now we do. Signed-off-by: Greg Farnum (cherry picked from commit 8e8892aa46accb519faa4bb9fecf66618f1b11b2) Conflicts: src/client/Client.cc (do not pass perm to get_quota_root() because jewel does not have 3caa4d233633fb7a67747f2c79c4a0ab89112294) commit dbe90c79b86743c7d143d33eb5389fcee0ac76ef Author: Mykola Golub Date: Tue Apr 11 22:31:43 2017 +0200 librbd: fix rbd_metadata_list and rbd_metadata_get - properly check for val_len in rbd_metadata_list - don't expect output buffers are zero pre-filled Fixes: http://tracker.ceph.com/issues/19588 Signed-off-by: Mykola Golub (cherry picked from commit 75afc74ea681402e22b6dec8b83276d145fc786b) commit 5096fc9c7c62e3043a9a0638eb2516792526fdd7 Author: Jason Dillaman Date: Wed Apr 12 10:47:28 2017 -0400 test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value Fixes: http://tracker.ceph.com/issues/19597 Signed-off-by: Jason Dillaman (cherry picked from commit 9ffd464dac102f684d6dfa78e58d2cb45e165ed6) commit 216156b5d4e2666ca592eaab2211940028422bc0 Author: Jason Dillaman Date: Fri Mar 10 10:56:38 2017 -0500 rbd: prevent adding multiple mirror peers to a single pool The rbd-mirror daemon does not currently support replication from multiple peers. Until that is supported, add a temporary restriction to prevent confusion. Fixes: http://tracker.ceph.com/issues/19256 Signed-off-by: Jason Dillaman (cherry picked from commit c0c9d1014d57b3d5b95e7513fcc38d04b9ea5165) commit c05bd1cb1f570c42b1999fbc70c86b0a72a7fcb3 Author: Orit Wasserman Date: Wed Apr 5 13:31:08 2017 +0300 radosgw-admin: use zone id when creating a zone Fixes: http://tracker.ceph.com/issues/19498 Signed-off-by: Orit Wasserman (cherry picked from commit 3fea36d635fcba8ca584a1c0ec9f07840009402c) commit ba81cbbdee1ccf95ceff56eef0a1b2b06be4024a Author: Casey Bodley Date: Tue Mar 14 15:43:13 2017 -0400 qa: rgw task uses period instead of region-map Signed-off-by: Casey Bodley (cherry picked from commit e3e3a71d1f1fb43bb4172ce2dfac9a28ca89df0f) commit a755c95262e10fac774716e460e2ba2eaee9df70 Author: Casey Bodley Date: Tue Mar 14 14:18:15 2017 -0400 rgw-admin: remove deprecated regionmap commands Fixes: http://tracker.ceph.com/issues/18725 Signed-off-by: Casey Bodley (cherry picked from commit 5830c1849a0c0110d17c37784808e456e6dcb7b3) Conflicts: src/rgw/rgw_admin.cc (trivial resolution) commit 482bd1adabf6f54c8f2bd73090d2e4c02d07f8a3 Author: hrchu Date: Wed Mar 29 02:17:04 2017 +0000 doc: rgw: correct the quota section Add the missing option and fix typo. Fixes: http://tracker.ceph.com/issues/19397 Signed-off-by: Chu, Hua-Rong (cherry picked from commit 51a88267f0d7f51aeb62092949b66b9f6c062e15) commit 21a83e1276e415e98a3780d0374bb9d4feb191d9 Author: Matt Benjamin Date: Wed Mar 15 16:35:16 2017 -0400 rgw_file: remove unused rgw_key variable Signed-off-by: Matt Benjamin (cherry picked from commit 1100a1c26e76485569cfebcf863b18cf908f6161) commit ebad040b3ab62e702afc52edd9d99d984b24cdc1 Author: Matt Benjamin Date: Fri Apr 14 15:56:37 2017 -0400 rgw_file: fix readdir after dirent-change Also, fixes link count computation off-by-one, update of state.nlink after computation, link computation reset at start, and a time print in debug log. Fixes: http://tracker.ceph.com/issues/19634 Signed-off-by: Matt Benjamin link count Signed-off-by: Matt Benjamin (cherry picked from commit e0f80266ecd424bf9466579b3edc03911a7c5719) commit dd9833cacaec20e5bd1a70ec46a427a7352eb5d6 Author: Matt Benjamin Date: Tue Apr 11 06:42:07 2017 -0400 rgw_file: don't expire directories being read If a readdir expire event turns out to be older than last_readdir, just reschedule it (but actually, we should just discard it, as another expire event must be in queue. Fixes: http://tracker.ceph.com/issues/19625 Signed-off-by: Matt Benjamin (cherry picked from commit 007b7451c26716c51207c161dc347e9a00da53f1) commit 30a5e857aade2474c5bd621c57938dfe333c3ae5 Author: Matt Benjamin Date: Wed Mar 15 16:40:35 2017 -0400 rgw_file: rgw_readdir: return dot-dirs only when *offset is 0 Signed-off-by: Matt Benjamin (cherry picked from commit 61482c2b85a07519f2256b1a3f2b6d8aa99d5f06) commit fe836bfb7a286c87c57576d29d6862be2514ada5 Author: Matt Benjamin Date: Tue Apr 11 05:56:13 2017 -0400 rgw_file: chunked readdir Adjust readdir callback path for new nfs-ganesha chunked readdir, including changes to respect the result of callback to not continue. Pending introduction of offset name hint, our caller will just be completely enumerating, so it is possible to remove the offset map and just keep a last offset. Fixes: http://tracker.ceph.com/issues/19624 Signed-off-by: Matt Benjamin (cherry picked from commit e0191d74e3aef06bf300df045a53a3952a71f651) commit 16eeb8c2acc8a1f843fa87967c2ee4c0ed2cd8bd Author: Gui Hecheng Date: Fri Mar 31 10:42:40 2017 +0800 rgw_file: fix missing unlock in unlink Fixes: http://tracker.ceph.com/issues/19435 Signed-off-by: Gui Hecheng (cherry picked from commit cb6808a6366a70f54d0cc16437d16aa1b7819c84) commit 8c7cb8227c39b09060cdca4e11dec75cf7f2336c Author: Matt Benjamin Date: Mon Mar 13 21:52:08 2017 -0400 rgw_file: implement reliable has-children check (unlink dir) Bug report and discussion provided by Gui Hecheng in nfs-ganesha upstream github. Briefly, while a reliable check is potentially costly, it is necessary. Fixes: http://tracker.ceph.com/issues/19270 Signed-off-by: Matt Benjamin (cherry picked from commit b05f1c6d61aa4501a971e87de6dcaf3e58c3d9b4) commit 18f14dd86400b50f46930a9ef56666d82035507e Author: Matt Benjamin Date: Tue Apr 4 20:16:13 2017 -0400 rgw_file: introduce rgw_lookup type hints The new type hints optimize object type deduction, when the rgw_lookup is called from an rgw_readdir callback. Fixes: http://tracker.ceph.com/issues/19623 Signed-off-by: Matt Benjamin (cherry picked from commit 2e66c7a7cc763c5c0d6f5db04855f60f2b2ceed3) commit 118ccad0ba763b98c9ee3de225a947be5372dc0d Merge: bf30ecd7cb 65465356b5 Author: Nathan Cutler Date: Thu Apr 20 11:08:45 2017 +0200 Merge pull request #14383 from smithfarm/wip-19547-jewel jewel: build/ops: rbdmap.service not included in debian packaging (jewel-only) Reviewed-by: Ken Dreyer commit bf30ecd7cb8dea43a9d55cb1b02c72bfda6bad09 Merge: 376c5e4753 9e123e6d6c Author: Nathan Cutler Date: Thu Apr 20 11:02:39 2017 +0200 Merge pull request #14143 from smithfarm/wip-19355-jewel jewel: rgw: when converting region_map we need to use rgw_zone_root_pool Reviewed-by: Casey Bodley commit 376c5e47531a2803ac9b7ea236b989c528268a9f Merge: da306df4de e2ee70a8ad Author: Nathan Cutler Date: Thu Apr 20 11:01:49 2017 +0200 Merge pull request #14136 from smithfarm/wip-19330-jewel jewel: rgw: upgrade to multisite v2 fails if there is a zone without zone info Reviewed-by: Casey Bodley commit da306df4deaaa782c0bad72539affd3ee1ec7efc Merge: 57b210da41 5ee8feaba4 Author: Nathan Cutler Date: Thu Apr 20 11:00:35 2017 +0200 Merge pull request #13872 from smithfarm/wip-19163-jewel jewel: doc: radosgw-admin: add the 'object stat' command to usage Reviewed-by: Casey Bodley commit 57b210da4185d624524d05bcd9ad01c2df16ca76 Merge: db92019e9a 9cd7dd8490 Author: Nathan Cutler Date: Thu Apr 20 10:59:33 2017 +0200 Merge pull request #13863 from smithfarm/wip-19155-jewel jewel: rgw: typo in rgw_admin.cc Reviewed-by: Casey Bodley commit db92019e9af43c2367fdf7865c0fab8cb38453e6 Merge: 6dcd5fa847 6add2a457e Author: Nathan Cutler Date: Thu Apr 20 10:56:35 2017 +0200 Merge pull request #13779 from smithfarm/wip-18866-jewel jewel: rgw: 'radosgw-admin sync status' on master zone of non-master zonegroup Reviewed-by: Casey Bodley commit eac0e27193c67ffca60b5e7c61e7769ad8ace6aa Author: Nathan Cutler Date: Wed Apr 19 16:53:57 2017 +0200 Revert "osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag." This reverts commit 2d68822c784eb4d62d3b0198ed4ec04404dbffb3. Signed-off-by: Nathan Cutler commit 0efe16d2566f0d6040f61fafd38c6661f08da1cd Author: Nathan Cutler Date: Wed Apr 19 16:53:49 2017 +0200 Revert "osdc/Objecter: resend RWORDERED ops on full" This reverts commit f2474042ecd6560323673170c13f2cb964406e70. Signed-off-by: Nathan Cutler commit 33af18e59298926d9636023faf871a191d582c1a Author: Nathan Cutler Date: Tue Apr 18 08:06:01 2017 +0200 doc: mention --show-mappings in crushtool manpage Fixes: http://tracker.ceph.com/issues/19649 Signed-off-by: Loic Dachary Signed-off-by: Nathan Cutler (cherry picked from commit b48b6f4ed8b5f5b5852cbbfd5b3d5b650efb7f1b) commit 6dcd5fa8474c3a43ffca2394557b11f4906ee1ff Merge: 7c006fc164 68fcb01211 Author: Gregory Farnum Date: Wed Apr 19 02:47:27 2017 -0400 Merge pull request #14596 from gregsfortytwo/wip-17916-divergent Wip 17916 divergent Reviewed-by: Josh Durgin commit 97cd21afc54efe3afb482b041f9c34ab6cdc682e Author: Nathan Cutler Date: Sun Apr 9 20:11:27 2017 +0200 tests: Thrasher: handle "OSD has the store locked" gracefully On slower machines (VPS, OVH) it takes time for the OSD to go down. Fixes: http://tracker.ceph.com/issues/19556 Signed-off-by: Nathan Cutler (cherry picked from commit a5b19d2d73540b730392f8001c8601f2cecc1b51) commit faeb8088ac3bd16bc04a1e5a55fec50285a7253f Author: Casey Bodley Date: Wed Apr 5 16:19:57 2017 -0400 rgw: fix for null version_id in fetch_remote_obj() commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f fixed the wrong use of the dest_obj's version, but removed the check for "null" version Signed-off-by: Casey Bodley (cherry picked from commit 915370776df5b964c2ee8d9f9329562919eef8d5) commit 6180fcb4ec33bdade37f5693193712f8016b2560 Author: Zhang Shaowen Date: Fri Mar 17 16:26:56 2017 +0800 rgw: version id doesn't work in fetch_remote_obj Signed-off-by: Zhang Shaowen (cherry picked from commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f) Conflicts: src/rgw/rgw_rados.cc: trivial: dest_obj.key.instance in master versus dest_obj.get_instance() in Jewel commit d079b91479abfb474fad4fafe54c119f009ea572 Author: liuchang0812 Date: Fri Feb 10 18:02:03 2017 +0800 rgw: don't return skew time in pre-signed url Fixes: http://tracker.ceph.com/issues/18828 Signed-off-by: liuchang0812 (cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb) commit 9b77b16b888b8efbf7d50d333e1880a6ec70d87a Author: Loic Dachary Date: Thu Apr 13 23:49:50 2017 +0200 ceph-disk: enable directory backed OSD at boot time https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6 introduced a regression preventing directory backed OSD from starting at boot time. For device backed OSD the boot sequence starts with ceph-disk@.service and proceeds to systemctl enable --runtime ceph-osd@.service where the --runtime ensure ceph-osd@12 is removed when the machine reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at boot time. However directory backed OSD solely rely on the ceph-osd@.service unit to start at boot time and will therefore fail to boot. The --runtime flag is selectively set for device backed OSD only. Fixes: http://tracker.ceph.com/issues/19628 Signed-off-by: Loic Dachary (cherry picked from commit f425a127b7487d2093c8c943f0bcdec3d673d601) Conflicts: src/ceph-disk/ceph_disk/main.py: trivial: Jewel does not support OpenRC and other inits, hence no corresponding 'elif' commit 68fcb01211e064f0d200cc9c9576254e9a6b949c Author: Greg Farnum Date: Mon Apr 17 18:09:55 2017 -0700 pglog: require users set a config option before ignoring divergent_priors Signed-off-by: Greg Farnum commit b9477303b010b3653934f77fa533df01aeff1c3c Author: Greg Farnum Date: Fri Apr 7 14:33:20 2017 -0700 osd: pglog: clean up divergent_priors off disk when running; don't assert on startup Fixes: http://tracker.ceph.com/issues/17916 Signed-off-by: Greg Farnum commit 7c006fc1640d2f6b9dc9002bfd994ecdf25510ee Merge: e31a540dce 721b2083cd Author: Gregory Farnum Date: Mon Apr 17 17:41:19 2017 -0400 Merge pull request #14492 from gregsfortytwo/wip-jewel-snaptrim Backport snap trimming improvements to Jewel Reviewed-by: Josh Durgin commit 721b2083cd2ed8567d41aba6b17caf30b678e6b3 Merge: a84dc8fe74 e31a540dce Author: Gregory Farnum Date: Mon Apr 17 17:39:59 2017 -0400 Merge branch 'jewel' into wip-jewel-snaptrim commit a84dc8fe747d1e17c7910c9857f7f5d3d56e2dff Author: Greg Farnum Date: Mon Apr 17 14:32:38 2017 -0700 PendingReleaseNotes: discuss snap trim improvements Signed-off-by: Greg Farnum commit 360a9d9af003c650cdf00534909d6488c702c413 Author: Greg Farnum Date: Wed Apr 12 16:30:55 2017 -0700 PrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine Rather than blocking the main op queue, just pause for that amount of time between state machine cycles. Also, add osd_snap_trim_sleep to a few of the thrasher yamls. Signed-off-by: Samuel Just (cherry picked from commit 2ed7759cfeb03e71f0fbd98fe7c2db2bb741861c) Conflicts: src/osd/PrimaryLogPG.cc Signed-off-by: Greg Farnum (cherry picked from commit 67336454a4cee66522bc0ca01b2c58b8960f75ec) Conflicts: qa/suites/rados/thrash/thrashers/default.yaml qa/suites/rados/thrash/thrashers/pggrow.yaml src/osd/OSD.h src/osd/ReplicatedPG.cc src/osd/ReplicatedPG.h Signed-off-by: Greg Farnum commit 18dbf6a0245e35dbbdb5ddb760182795b37983c0 Author: Samuel Just Date: Thu Jan 26 15:41:21 2017 -0800 rados: check that pool is done trimming before removing it Signed-off-by: Samuel Just (cherry picked from commit 4aebf59d906fa3e03d21bdac182f89fe3cd4c802) (cherry picked from commit 34398c29b3c57f00d932cf96570f882dce64a82b) commit 7f78450bd184335ba0098f3ded0fefa2c79a5dd5 Author: Greg Farnum Date: Tue Apr 11 14:04:19 2017 -0700 osd/ReplicatedPG: limit the number of concurrently trimming pgs This patch introduces an AsyncReserver for snap trimming to limit the number of pgs on any single OSD which can be trimming, as with backfill. Unlike backfill, we don't take remote reservations on the assumption that the set of pgs with trimming work to do is already well distributed, so it doesn't seem worth the implementation overhead to get reservations from the peers as well. Signed-off-by: Samuel Just (cherry picked from commit 21cc515adfb225ba70f1d80b1b76f0345c214c22) Conflicts: src/osd/PrimaryLogPG.cc src/osd/PrimaryLogPG.h Signed-off-by: Greg Farnum (cherry picked from commit 68ea24396ca6450d4d8042a7c5f51306b7d199fa) (cherry picked from commit c7176b869898c870b56b1762958652d801af4c4c) Conflicts: Many. As evidenced by involving two distinct patches in this one commit, it wasn't a clean backport. Signed-off-by: Greg Farnum commit 527911fab78b4752313a4a2a5d3ab0ae736bc50f Author: Kefu Chai Date: Mon Apr 10 14:53:46 2017 +0800 mon/MonClient: make get_mon_log_message() atomic * LogClient: move reset_session() into get_mon_log_message() and add a "flush" param to the latter. so it can get_mon_log_message() atomically. otherwise another call changing the log queue could sneak in between reset_session() and get_mon_log_message(). * MonClient: add a "flush" param to do_send() so we can reset the LogClient session once we are connected to a monitor. Fixes: http://tracker.ceph.com/issues/19427 Signed-off-by: Kefu Chai (cherry picked from commit 5215e291da2b527d85e129eda86043490843178e) Conflicts: src/mon/MonClient.cc: handle_auth: replaced 'log_client->reset_session(); send_log();' sequence with newly introduced 'send_log(true);' like the original patch does commit e31a540dcea96b3d5b4f7ecd20e2d54e81a68e2b Merge: 7c36d1650f 06916a8798 Author: Nathan Cutler Date: Fri Apr 14 22:28:51 2017 +0200 Merge pull request #13834 from smithfarm/wip-18969-jewel jewel: rgw: Change loglevel to 20 for 'System already converted' message Reviewed-by: Casey Bodley Reviewed-by: Nathan Cutler commit 7c36d1650fdb7a357ff724b37cf59b4da413f76b Merge: 0e3aa2cb01 4c1f302f7d Author: Nathan Cutler Date: Fri Apr 14 22:26:51 2017 +0200 Merge pull request #13833 from smithfarm/wip-18908-jewel jewel: rgw: the swift container acl does not support field .ref Reviewed-by: Radoslaw Zarzynski commit b698d1fa4ce4aca5e392eeec600e3357c3cf71a9 Author: Jason Dillaman Date: Thu Mar 16 12:28:41 2017 -0400 librbd: is_exclusive_lock_owner API should ping OSD This is required to detect if a peer has been silently blacklisted and is therefore no longer the lock owner. Fixes: http://tracker.ceph.com/issues/19287 Signed-off-by: Jason Dillaman (cherry picked from commit e15db05960a284bdf3701256722299d553cfd5aa) Conflicts: src/librbd/ManagedLock.[h|cc]: logic moved to ExclusiveLock (cherry picked from commit 7e30b630e2806c73ea503871599f958b58df7934) commit 0e3aa2cb011be8a2af69040b94ee240ea6c1e663 Merge: 8d5a5ddfec 419c9926d9 Author: Nathan Cutler Date: Thu Apr 13 11:09:27 2017 +0200 Merge pull request #13214 from ovh/bp-osd-updateable-throttles-jewel jewel: osd: allow client throttler to be adjusted on-fly, without restart Reviewed-by: Josh Durgin commit 8d5a5ddfecc7c020eeb8aeb927e007bc04885534 Merge: 091aaa2ab7 bcd3c906e5 Author: Nathan Cutler Date: Thu Apr 13 10:44:23 2017 +0200 Merge pull request #14326 from shinobu-x/wip-15025-jewel jewel: osd: new added OSD always down when full flag is set Reviewed-by: Josh Durgin commit d311eea6bc5b2c88984cfc16340970ea1459b74a Author: Robin H. Johnson Date: Sat Feb 11 10:32:53 2017 -0800 msg/simple/Pipe: support IPv6 QoS. Extend DSCP marking for heartbeat packets to IPv6, as commit 9b9a682fe035c985e416ee1c112fa58f9045a27c only implemented support for IPv4. Conflicts: Cherry-picked 91a29bc490fdfbbef0875fa620c7ba1a1a6492ae from master to avoid conflict. Backport: jewel, luminious Fixes: http://tracker.ceph.com/issues/18887 Signed-off-by: Robin H. Johnson (cherry picked from commit 2d6021fbf7a728f73c2998be17e9224f14b83a30) commit 332b5174c769f395074255e075de8d2cc1ee4021 Author: Jason Dillaman Date: Thu Mar 16 12:27:08 2017 -0400 pybind: fix incorrect exception format strings Signed-off-by: Jason Dillaman (cherry picked from commit 68617455f534a612ade1331f43b032ab524704ae) commit 091aaa2ab768858e840e2d05e0896c229ce69984 Merge: 3f2e4cd2d5 d30c4d55ad Author: Nathan Cutler Date: Wed Apr 12 11:38:48 2017 +0200 Merge pull request #13874 from smithfarm/wip-19171-jewel jewel: doc: rgw S3 create bucket should not do response in json Reviewed-by: Abhishek Lekshmanan commit 3f2e4cd2d53f1cbdbdb1fc8687c5ffe0d46b346b Merge: ea0bc6c553 0e11a938c5 Author: Nathan Cutler Date: Wed Apr 12 11:06:40 2017 +0200 Merge pull request #13492 from shinobu-x/wip-18516-jewel jewel: build/ops: "osd marked itself down" will not recognised if host runs mon + osd on shutdown/reboot Reviewed-by: Nathan Cutler commit ea0bc6c553454f7641a2594013412aee142fbd11 Merge: 845972f4de d012c381e8 Author: Nathan Cutler Date: Wed Apr 12 11:01:23 2017 +0200 Merge pull request #13254 from shinobu-x/wip-14609-jewel jewel: common: radosstriper: protect aio_write API from calls with 0 bytes Reviewed-by: Kefu Chai commit 845972f4de1d803aa4dab0e1afaed693bbba088f Merge: a3deef997f cfa37d6a16 Author: Nathan Cutler Date: Wed Apr 12 10:52:10 2017 +0200 Merge pull request #13489 from shinobu-x/wip-18955-jewel jewel: ceph-disk: bluestore --setgroup incorrectly set with user Reviewed-by: Loic Dachary Reviewed-by: Nathan Cutler commit a3deef997ff0800b9e3e2d141cfc6fdc73cac837 Merge: 702edb5519 39aab763a4 Author: Nathan Cutler Date: Wed Apr 12 10:51:42 2017 +0200 Merge pull request #14070 from smithfarm/wip-19339-jewel jewel: tests: dummy suite fails in OpenStack Reviewed-by: Loic Dachary commit 702edb5519e67bc5f8c5b65c6f63c9635cd758cf Merge: f509ccc4b7 a20d2b89ee Author: Nathan Cutler Date: Wed Apr 12 10:51:12 2017 +0200 Merge pull request #14329 from smithfarm/wip-19493-jewel jewel: ceph-disk: Racing between partition creation & device node creation Reviewed-by: Loic Dachary Reviewed-by: Sébastien Han commit f509ccc4b734697267cec35ff57dd7f1c5aaaddb Merge: c8c4bff2af f2474042ec Author: Nathan Cutler Date: Wed Apr 12 10:48:29 2017 +0200 Merge pull request #14427 from smithfarm/wip-19140-jewel jewel: osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag Reviewed-by: Kefu Chai commit c8c4bff2afa7fe0339a97f32e69bb9d0546f1318 Merge: 349baea1a4 b5b441abaa Author: Nathan Cutler Date: Wed Apr 12 10:48:08 2017 +0200 Merge pull request #14324 from shinobu-x/wip-19371-jewel jewel: common: monitor creation with IPv6 public network segfaults Reviewed-by: Kefu Chai commit 349baea1a4486e475e0381a6b316d64a6ce0139c Merge: dd466b7d9a 72e2476a13 Author: Nathan Cutler Date: Wed Apr 12 10:45:35 2017 +0200 Merge pull request #14112 from shinobu-x/wip-19192-jewel jewel: tools: brag fails to count "in" mds Reviewed-by: Kefu Chai commit dd466b7d9acb03c8830bdd83b3b73602c68083c2 Merge: b8f8bd0a94 ee06517547 Author: Nathan Cutler Date: Wed Apr 12 10:45:11 2017 +0200 Merge pull request #14150 from smithfarm/wip-18823-jewel jewel: tests: run-rbd-unit-tests.sh assert in lockdep_will_lock, TestLibRBD.ObjectMapConsistentSnap Reviewed-by: Jason Dillaman commit b8f8bd0a949c917b119eca91aec95c6a971a1fb4 Merge: 222916a375 1cc8d0d085 Author: Nathan Cutler Date: Wed Apr 12 10:44:47 2017 +0200 Merge pull request #14152 from smithfarm/wip-18893-jewel jewel: librbd: Incomplete declaration for ContextWQ in librbd/Journal.h Reviewed-by: Jason Dillaman commit 222916a3758b0253bcb851b5e955f6970f171306 Merge: 49f84b1a14 b85677397e Author: Nathan Cutler Date: Wed Apr 12 10:44:25 2017 +0200 Merge pull request #14154 from smithfarm/wip-18948-jewel jewel: tests: additional rbd-mirror test stability improvements Reviewed-by: Jason Dillaman commit 49f84b1a14295e80ef35204ed691b3131c5f744c Merge: 2a232d43a1 f6489d01ca Author: Nathan Cutler Date: Wed Apr 12 10:44:04 2017 +0200 Merge pull request #14148 from smithfarm/wip-18778-jewel jewel: rbd: rbd --pool=x rename y z does not work Reviewed-by: Jason Dillaman commit 2a232d43a18b47399f96abeac3ac800f204a9eaf Merge: 413ac584d6 8bed107b84 Author: Nathan Cutler Date: Wed Apr 12 10:43:41 2017 +0200 Merge pull request #14083 from smithfarm/wip-19357-jewel jewel: rbd: systemctl stop rbdmap unmaps all rbds and not just the ones in /etc/ceph/rbdmap Reviewed-by: Jason Dillaman commit 413ac584d63fd469ea28defa68c39538444d01b1 Merge: 23d595b1b0 547e867628 Author: Nathan Cutler Date: Wed Apr 12 10:43:21 2017 +0200 Merge pull request #13154 from smithfarm/wip-18496-jewel jewel: librbd: Possible deadlock performing a synchronous API action while refresh in-progress Reviewed-by: Jason Dillaman commit 23d595b1b0fb874adfd1507d86db261cf6a193a9 Merge: 4add6f5580 915dbace5d Author: Nathan Cutler Date: Wed Apr 12 10:42:54 2017 +0200 Merge pull request #13244 from smithfarm/wip-18775-jewel jewel: rbd: qemu crash triggered by network issues Reviewed-by: Jason Dillaman commit 4add6f5580b7d65571bc426e95fac35be7150ff4 Merge: 37ab19cc29 1a4e1e09b1 Author: Nathan Cutler Date: Wed Apr 12 10:42:23 2017 +0200 Merge pull request #13809 from asheplyakov/18321-bp-jewel jewel: librbd: remove image header lock assertions Reviewed-by: Jason Dillaman commit 37ab19cc29d61970e08af1b4627137cfcaa99474 Merge: f7c04e3ca6 8d0140a9ed Author: Nathan Cutler Date: Wed Apr 12 10:41:56 2017 +0200 Merge pull request #13107 from smithfarm/wip-18669-jewel jewel: tests: [ FAILED ] TestLibRBD.ImagePollIO in upgrade:client-upgrade-kraken-distro-basic-smithi Reviewed-by: Jason Dillaman commit f7c04e3ca69c7fc134b267e825277eccea228c9b Merge: d2909bdb4c a18a2dd108 Author: Nathan Cutler Date: Wed Apr 12 10:39:26 2017 +0200 Merge pull request #13585 from asheplyakov/jewel-bp-16585 jewel: msg: set close on exec flag Reviewed-by: Kefu Chai commit d2909bdb4cf135de850cc865f735cc61eb3d4ea6 Merge: cd748603da 6d47615c11 Author: Nathan Cutler Date: Wed Apr 12 03:56:22 2017 +0200 Merge pull request #14371 from tchaikov/wip-19429-jewel jewel: tests: clone workunit using the branch specified by task Nathan Cutler commit cd748603da43c0b7377b5ef07aff618711b4879c Merge: 1a20c12355 dd25a8f36b Author: Nathan Cutler Date: Wed Apr 12 03:54:11 2017 +0200 Merge pull request #14325 from shinobu-x/wip-18619-jewel jewel: osd: degraded and misplaced status output inaccurate Reviewed-by: Josh Durgin commit 1a20c123556e2f566af7e87ab3da6fc2ca328bb9 Merge: 4838c4db4c 7fdf4d41c5 Author: Nathan Cutler Date: Wed Apr 12 03:53:25 2017 +0200 Merge pull request #14236 from smithfarm/wip-19392-jewel jewel: mon: remove bad rocksdb option Reviewed-by: Josh Durgin commit 4838c4db4c610926edc599677cb7e74f3bdc3077 Merge: e26b7033eb 3860ccf16d Author: Nathan Cutler Date: Wed Apr 12 03:52:41 2017 +0200 Merge pull request #14181 from mslovy/wip-19394-jewel jewel: osd: bypass readonly ops when osd full Reviewed-by: Josh Durgin commit e26b7033ebd4a09210d482a54677806fe2c03505 Merge: 389150bcc3 819af9e413 Author: Nathan Cutler Date: Wed Apr 12 03:49:49 2017 +0200 Merge pull request #14113 from shinobu-x/wip-19319-jewel jewel: cli: RadosImport::import should return an error if Rados::connect fails Reviewed-by: Brad Hubbard Reviewed-by: David Zafman commit 389150bcc376702022cf15c0c79dbe856a97dffa Merge: a8b10082f5 90de64bd81 Author: Nathan Cutler Date: Wed Apr 12 03:45:33 2017 +0200 Merge pull request #14047 from asheplyakov/reindex-on-pg-split jewel: osd: reindex properly on pg log split Reviewed-by: Sage Weil Reviewed-by: Josh Durgin commit a8b10082f5761ea1132b85bd916a84f87e0276e2 Merge: 32ed9b7897 ae498e84ff Author: Nathan Cutler Date: Wed Apr 12 03:45:15 2017 +0200 Merge pull request #14044 from mslovy/wip-19311-jewel jewel: core: os/filestore: fix clang static check warn use-after-free Reviewed-by: Sage Weil Reviewed-by: Josh Durgin commit 32ed9b789733e21a2d597c7016eabf95763e50ae Merge: 6705e911a2 335b5fa4a9 Author: Nathan Cutler Date: Wed Apr 12 03:44:25 2017 +0200 Merge pull request #13932 from asheplyakov/18911-bp-jewel jewel: rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly Reviewed-by: Mykola Golub Reviewed-by: Jason Dillaman commit 6705e911a229cee50d5ac992c7dd9af834f2400d Merge: 3d21a0080d 714eb863c3 Author: Nathan Cutler Date: Wed Apr 12 03:42:13 2017 +0200 Merge pull request #13831 from jan--f/wip-19206-jewel jewel: fs: Invalid error code returned by MDS is causing a kernel client WARNING Reviewed-by: John Spray commit 3d21a0080dfd00c0b89ed23fd8049e46c3041af4 Merge: 8a6d64331b 6b5322c5f6 Author: Nathan Cutler Date: Wed Apr 12 03:41:36 2017 +0200 Merge pull request #13827 from tchaikov/wip-19185-jewel jewel: osd: ReplicatedPG: try with pool's use-gmt setting if hitset archive not found Reviewed-by: Sage Weil Reviewed-by: Josh Durgin commit 8a6d64331b73a4a24a59f4dcdb6771f1dc638879 Merge: f96392a589 cebba011e5 Author: Nathan Cutler Date: Wed Apr 12 03:40:09 2017 +0200 Merge pull request #13788 from shinobu-x/wip-18235-jewel jewel: core: os/filestore/HashIndex: be loud about splits Reviewed-by: Josh Durgin commit f96392a58970ab1e81653d5657d3d680120a02b6 Merge: 8fe6ffcfef 1d054c3856 Author: Nathan Cutler Date: Wed Apr 12 03:39:27 2017 +0200 Merge pull request #13786 from shinobu-x/wip-19129-jewel jewel: build/ops: ceph-base package missing dependency for psmisc Reviewed-by: Nathan Cutler commit 8fe6ffcfef8918fd8634e87255ad3fe7a544aa16 Merge: 6f589a159e b249fd5bd8 Author: Nathan Cutler Date: Wed Apr 12 03:38:34 2017 +0200 Merge pull request #13732 from liewegas/wip-19119-jewel jewel: doc: PendingReleaseNotes: note about 19119 Reviewed-by: Nathan Cutler commit 6f589a159e4eb5f6c96634a742acaf6cd6e174c2 Merge: b8f2d35884 8c7a1df251 Author: Nathan Cutler Date: Wed Apr 12 03:36:21 2017 +0200 Merge pull request #13541 from shinobu-x/wip-18929-jewel jewel: osd: restrict want_acting to up+acting on recovery completion Reviewed-by: Josh Durgin Reviewed-by: Nathan Cutler commit b8f2d35884a52586a28f1ff4eaf99c8c3ba1c43f Merge: 40d1443d68 2d17092fab Author: Nathan Cutler Date: Wed Apr 12 03:34:54 2017 +0200 Merge pull request #13477 from asheplyakov/jewel-bp-18951 jewel: osd: --flush-journal: sporadic segfaults on exit Reviewed-by: Sage Weil Reviewed-by: Josh Durgin commit 40d1443d68b788737f83c10db3811bc37dbe8dda Merge: 50e863e0f4 07501dec6f Author: Nathan Cutler Date: Wed Apr 12 03:33:25 2017 +0200 Merge pull request #13261 from shinobu-x/wip-18587-jewel jewel: mon: OSDMonitor: make 'osd crush move ...' work on osds Reviewed-by: Josh Durgin Reviewed-by: Nathan Cutler commit 25e43ac25619d883d5a04e2df1cf6f57fea73fd0 Author: Nathan Cutler Date: Tue Apr 11 08:23:23 2017 +0200 tests: fix oversight in yaml comment When the file was copied from the hammer version, the word "hammer" was not changed to "infernalis". Signed-off-by: Nathan Cutler This cannot be cherry-picked from master because the test has been dropped. commit f2474042ecd6560323673170c13f2cb964406e70 Author: Sage Weil Date: Thu Mar 2 21:20:08 2017 -0600 osdc/Objecter: resend RWORDERED ops on full Our condition for respecting the FULL flag is complex, and involves the WRITE | RWORDERED flags vs the FULL_FORCE | FULL_TRY flags. Previously, we could block a read bc of RWORDRED but not resend it later. Fix by capturing the complex condition in a respects_full() bool and using it both for the blocking-on-send and resending-on-possibly-notfull-later checks. Fixes: http://tracker.ceph.com/issues/19133 Signed-off-by: Sage Weil (cherry picked from commit c4b73f19a7be13ff412eef804efcd8c18ed4dae6) commit 2d68822c784eb4d62d3b0198ed4ec04404dbffb3 Author: Jianpeng Ma Date: Thu May 5 23:44:57 2016 +0800 osdc/Objecter: If osd full, it should pause read op which w/ rwordered flag. Signed-off-by: Jianpeng Ma (cherry picked from commit 07b2a22210e26eac1b2825c30629788da05e5e12) commit 2271cd81282f3f026316134cbab630f3e4f47782 Author: Brad Hubbard Date: Mon Apr 3 13:37:17 2017 +1000 ceph-disk: Populate mount options when running "list" Also tidy up by moving duplicated code into a function Fixes: http://tracker.ceph.com/issues/17331 Signed-off-by: Brad Hubbard (cherry picked from commit 7943ab2e01e24f2dfc5b6f1d3ffdc8a49e01af45) commit 65465356b5e7cb6cba67f8cbb81259d21e888dfb Author: Ken Dreyer Date: Tue Feb 16 12:56:34 2016 -0700 debian: replace SysV rbdmap with systemd service Stop shipping /etc/init.d/rbdmap in the Debian packages. Ship the rbdmap.service systemd unit file instead. The corresponding change has already been made for RPMs, in 9224ac2ad25f7d017916f58b642c0ea25305c3e5. For Upstart-based systems (eg Ubuntu Trusty), the Debian packages already contain rbdmap.conf. (This gets us a tiny bit closer to being able to remove the rbdmap SysV script from our tree entirely.) Signed-off-by: Ken Dreyer (cherry picked from commit 839807118dda2fb4d57ed9d50ec46e3ee0e2820a) Conflicts: debian/ceph-common.install (retain /etc/init.d/rbdmap so jewel users can choose sysvinit or systemd) debian/rules (retain /etc/init.d/rbdmap so jewel users can choose sysvinit or systemd) commit 6d47615c11a216733ba368f0dbd9a0a1b9c8fd35 Author: Kefu Chai Date: Sat Apr 1 23:04:22 2017 +0800 qa/tasks/workunit.py: use "overrides" as the default settings of workunit otherwise the settings in "workunit" tasks are always overridden by the settings in template config. so we'd better follow the way of how "install" task updates itself with the "overrides" settings: it uses the "overrides" as the *defaults*. Fixes: http://tracker.ceph.com/issues/19429 Signed-off-by: Kefu Chai (cherry picked from commit 47080150a17d238f38d9da824d227393ad767aad) commit fdc71e75cd6361be49c4c20e77d3bdff017b38bf Author: Kefu Chai Date: Thu Mar 30 12:37:01 2017 +0800 tasks/workunit.py: specify the branch name when cloning a branch c1309fb failed to specify a branch when cloning using --depth=1, which by default clones the HEAD. and we can not "git checkout" a specific sha1 if it is not HEAD, after cloning using '--depth=1', so in this change, we dispatch "tag", "branch", "HEAD" using three Refspec classes. Signed-off-by: Kefu Chai Signed-off-by: Dan Mick (cherry picked from commit 9ca7ccf5f1739f731da8bf31260594aea3a2932d) Conflicts: qa/tasks/workunit.py (trivial resolution) commit 6c14a803894d71bae858705855551a44bdf9bf67 Author: Dan Mick Date: Tue Mar 28 20:08:13 2017 -0700 tasks/workunit.py: when cloning, use --depth=1 Help avoid killing git.ceph.com. A depth 1 clone takes about 7 seconds, whereas a full one takes about 3:40 (much of it waiting for the server to create a huge compressed pack) Signed-off-by: Dan Mick (cherry picked from commit c1309fbef300a062138ac40eb5d3e5081b833072) commit f8aa6be06cc628b1cf64c9196c30045d020d657e Author: Nathan Cutler Date: Fri Apr 7 06:59:13 2017 +0200 build/ops: rpm: move $CEPH_EXTRA_CONFIGURE_ARGS to right place Signed-off-by: Nathan Cutler commit bb3eb4284fe74cbf78e4a406c7b5f67a8e3c84b3 Author: Nathan Cutler Date: Fri Apr 7 06:48:51 2017 +0200 build/ops: rpm: explicitly provide --with-ocf to configure Fixes: http://tracker.ceph.com/issues/19546 Signed-off-by: Nathan Cutler (Note: This cannot be cherry-picked because master uses cmake, but the fix does bring the jewel spec file into better alignment its master counterpart, at least as far as this one little bit is concerned.) commit 30c952735327d05b6049e9364c1a053ebf651751 Author: lu.shasha Date: Mon Feb 27 15:52:43 2017 +0800 rgw: use separate http_manager for read_sync_status concurrent users of read_sync_status() use different cr managers, when get_resource must use http_manager related to the cr manager. Fixes: http://tracker.ceph.com/issues/19236 Signed-off-by: Shasha Lu (cherry picked from commit c412024889f8995d98096ac863bafee71624bd70) commit 87cb8474f523be1b281882c46a8a3597977a51c9 Author: Casey Bodley Date: Thu Jul 28 20:20:29 2016 -0400 rgw: pass cr registry to managers Signed-off-by: Casey Bodley (cherry picked from commit ef4d7eab11fb5d2a41c9c28b9eb8b075aaff0d05) Conflicts: src/rgw/rgw_rados.cc: removed ref to RGWSyncLogTrimThread (not backported) commit 1a6d7c0506d4bcda775dda05bb357d4d5695dabb Author: Casey Bodley Date: Fri Jul 22 11:00:16 2016 -0400 rgw: use separate cr manager for read_sync_status RGWCoroutinesManager::run() is not reentrant, so concurrent users of read_sync_status() must use different managers Signed-off-by: Casey Bodley (cherry picked from commit 6b1e40d7a21c549b55e6576ec56973c8d3c092d1) commit c466adee9c8f7a97ff7e99bee56da8ce51bf0f00 Author: Casey Bodley Date: Thu Jul 21 23:46:20 2016 -0400 rgw: change read_sync_status interface RGWDataSyncStatusManager::read_sync_status() now operates on the given parameter, rather than its internal member variable. this allows multiple concurrent readers, which is needed for the rest interface Signed-off-by: Casey Bodley (cherry picked from commit b7cd4e0e8b879b5e528de75bea3307585b96cbf2) commit 36921a3f3d01547508dc6270f6b19f2576e067d9 Author: Casey Bodley Date: Wed Jun 8 11:24:11 2016 -0400 rgw: don't ignore ENOENT in RGWRemoteDataLog::read_sync_status() rest handlers for sync status need to return ENOENT errors. the only other callers are in radosgw-admin, so the ENOENT errors are ignored at those call sites instead Signed-off-by: Casey Bodley (cherry picked from commit 2cc533b30c0f23c0750ea8d02c51b3b3d3b4821a) commit b249fd5bd816a63b445db12c8f846cfda199c8b8 Author: Sage Weil Date: Wed Mar 1 13:18:44 2017 -0600 PendingReleaseNotes: warning about 'osd rm ...' and #19119 See http://tracker.ceph.com/issues/19119 Signed-off-by: Sage Weil (cherry picked from commit be96003c464481d8e84825178d600234a0d64d22) commit 335258f975a8e8539774e8cb22690d746ec90d9f Author: Ilya Dryomov Date: Tue Mar 28 11:49:08 2017 +0200 osdc/Objecter: respect epoch barrier in _op_submit() Epoch barrier instructs us to avoid sending (i.e. pause) any OSD ops until we see a barrier epoch. The only thing epoch_barrier check in target_should_be_paused() does is keep already paused ops paused. We need to actually pause incoming OSD ops in _op_submit(). Fixes: http://tracker.ceph.com/issues/19396 Signed-off-by: Ilya Dryomov (cherry picked from commit f8e8efc0a53d7bd807cc0c2178aef7c4bed62ab7) commit a20d2b89ee13e311cf1038c54ecadae79b68abd5 Author: Erwan Velu Date: Fri Mar 31 14:54:33 2017 +0200 ceph-disk: Adding retry loop in get_partition_dev() There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/. It appear that waiting a very short is usually enough to get the partition beein populated. Analysis: update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and the actual creation on the /sys/block//* entrypoint. On our CI that race occurs pretty often but trying to reproduce it locally never been possible. This patch is almost a workaround rather than a fix to the real problem. It offer retrying after a very short to be make a chance the device to appear. This approach have been succesful on the CI. Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs. A typical output from the build running on a CI with that code. command_check_call: Running command: /usr/bin/udevadm settle --timeout=600 get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda get_partition_dev: Found partition 2 for /dev/sda after 1 tries get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid fixes: #19428 Signed-off-by: Erwan Velu (cherry picked from commit 93e7b95ed8b4c78daebf7866bb1f0826d7199075) commit 2d5d0aec60ec9689d44a53233268e9b9dd25df95 Author: Erwan Velu Date: Wed Mar 22 10:11:44 2017 +0100 ceph-disk: Reporting /sys directory in get_partition_dev() When get_partition_dev() fails, it reports the following message : ceph_disk.main.Error: Error: partition 2 for /dev/sdb does not appear to exist The code search for a directory inside the /sys/block/get_dev_name(os.path.realpath(dev)). The issue here is the error message doesn't report that path when failing while it might be involved in. This patch is about reporting where the code was looking at when trying to estimate if the partition was available. Signed-off-by: Erwan Velu (cherry picked from commit 413c9fcfbe8e6ab33d73b8428090ccacc33c5d15) commit bcd3c906e5b57e7f44df1963c6e11b78ff89482c Author: Mingxin Liu Date: Mon Mar 13 23:41:58 2017 +0800 osd: don't share osdmap with objecter when preboot Signed-off-by: Mingxin Liu (cherry picked from commit a5a3644eecc49b4eea890c6999fe87536495dcbe) commit dd25a8f36bef1901f3ce6193cfcbdaf7ab2424a1 Author: David Zafman Date: Wed Jan 18 08:33:40 2017 -0800 osd: Calculate degraded and misplaced more accurately Calculate num_object_copies based on the larger of pool size, up set size and acting set size. Calculate num_objects_degraded as the difference between num_object_copies and all copies found on acting set and backfilling up set OSDs. Calculate num_objects_misplaced as all copies on acting set OSDs not in up set less copies that have been backfilled to up set OSDs. Fixes: http://tracker.ceph.com/issues/18619 Signed-off-by: David Zafman (cherry picked from commit 8423bc40759cca137f61e7b755411719a84369d4) commit b5b441abaa852e85ddefd8b22835c9b85898cc06 Author: Fabian Grünbichler Date: Wed Mar 22 16:13:50 2017 +0100 common: fix segfault in public IPv6 addr picking sockaddr is only 16 bytes big, so declaring net as sockaddr and then casting to sockaddr_in6 in case of IPv6 cannot work. using sockaddr_storage works for both IPv4 and IPv6, and is used in other code parts as well. note that the tests did not find this issue as they declared the bigger structs and casted the references to (sockaddr *) Fixes: http://tracker.ceph.com/issues/19371 Signed-off-by: Fabian Grünbichler (cherry picked from commit ae2ee3d3835fe25b35eeb1a841ee5234cd69eb65) commit 7fdf4d41c5bef14269cb302301a08d5a3a57a768 Author: Sage Weil Date: Mon May 2 15:29:12 2016 -0400 mon: remove bad rocksdb option Signed-off-by: Sage Weil (cherry picked from commit 0ac671ece258e509f71a05253e62a9878e279840) commit 3860ccf16d7dfb137face9886c3d7f29cd527835 Author: Jianpeng Ma Date: Thu May 5 23:07:06 2016 +0800 osd: bypass readonly ops when osd full. Signed-off-by: Jianpeng Ma (cherry picked from commit e2a0ae8e88e6b7354b14adb503fd8ba8525bee39) See: http://tracker.ceph.com/issues/19394 Signed-off-by: yaoning commit b85677397ef9a3fe16c087e67d3f752851bbe070 Author: Jason Dillaman Date: Mon Nov 21 15:31:43 2016 -0500 qa/workunits/rbd: resolve potential rbd-mirror race conditions Fixes: http://tracker.ceph.com/issues/18935 Signed-off-by: Jason Dillaman (cherry picked from commit 63eae97afc1a92412525468263fb8696a243ebac) commit 1cc8d0d08560af4e8785d7c4c6a925b1fc9f988e Author: Boris Ranto Date: Wed Feb 8 23:47:57 2017 +0100 librbd: Include WorkQueue.h since we use it We use m_work_queue of type ContextWQ in handle_update function but we do not include common/WorkQueue.h that defines ContextWQ. This results in dereference of an incomplete type and causes build error in latest Fedora rawhide (future 26). Fixes: http://tracker.ceph.com/issues/18862 Signed-off-by: Boris Ranto (cherry picked from commit 480f82847ad1fc7959f1fe5a90761a5a24550993) commit ee06517547ae174472d739f966c0a27d3a97d742 Author: Jason Dillaman Date: Wed Jan 18 20:54:22 2017 -0500 librbd: avoid possible recursive lock when racing acquire lock Fixes: http://tracker.ceph.com/issues/17447 Signed-off-by: Jason Dillaman (cherry picked from commit 5e46e8eb664f573bd70ae7c96a6d9a98b0deb09e) commit f6489d01ca41d6979b5de28e3cde6b43fcaa8edb Author: Gaurav Kumar Garg Date: Mon Jan 30 13:03:20 2017 +0100 rbd: destination pool should be source pool if it is not specified Currently if user perform image rename operation and user give pool name as a optional parameter (--pool=) then currently its taking this optional pool name for source pool and making destination pool name default pool name. With this fix if user provide pool name as a optional pool name parameter then it will consider both soruce and destination pool name as optional parameter pool name. Fixes: http://tracker.ceph.com/issues/18326 Reported-by: МАРК КОРЕНБЕРГ Signed-off-by: Gaurav Kumar Garg (cherry picked from commit 01f23aa99fb694da326ab408e75b33c640ce660b) commit 9e123e6d6c8bbd54514b498df5a22d961f0cefbb Author: Orit Wasserman Date: Sun Mar 12 12:11:28 2017 +0200 rgw: use rgw_zone_root_pool for region_map like is done in hammer Fixes: http://tracker.ceph.com/issues/19195 Signed-off-by: Orit Wasserman (cherry picked from commit c91dd6d9efd148e0fe0f027dde537e977de9aa26) commit e2ee70a8ad51992bbd763d2465f6d8a01dad6a31 Author: Orit Wasserman Date: Thu Mar 9 13:03:24 2017 +0200 rgw: skip conversion of zones without any zoneparams Fixes: http://tracker.ceph.com/issues/19231 Signed-off-by: Orit Wasserman (cherry picked from commit 36cf5a5c8179c6313346b2e29286c537c6fefce8) commit c7d292bf6714d7aaf10412e5109badb90f5dc208 Author: Orit Wasserman Date: Thu Mar 9 11:16:26 2017 +0200 rgw: better debug information for upgrade Signed-off-by: Orit Wasserman (cherry picked from commit e9f3bf8eab1dd46a92f54b0f7afe1f4c0e4204db) commit 11f5c841c7698e6239017478fa05f742b7c0ab1c Author: Danny Al-Gaaf Date: Tue Jan 31 18:01:32 2017 +0100 rgw/rgw_rados.cc: prefer ++operator for non-primitive iterators Signed-off-by: Danny Al-Gaaf (cherry picked from commit 7086cf9a73f2ec1eb96c0e752beb1b74fca18570) commit 819af9e4139997cd845dc24a137d43218d8a40a8 Author: Brad Hubbard Date: Tue Mar 21 12:22:20 2017 +1000 tools/rados: Check return value of connect Fail gracefully if Rados::connect returns an error. Fixes: http://tracker.ceph.com/issues/19319 Signed-off-by: Brad Hubbard (cherry picked from commit c119091ef0844e4a1ddd790a8bfef8f06bb57d58) commit 72e2476a130e14abcd541ff61328454cb69ad9c3 Author: Kefu Chai Date: Mon Mar 6 11:33:27 2017 +0800 brag: count the number of mds in fsmap not in mdsmap this change was introduced in 4e9b953 Fixes: http://tracker.ceph.com/issues/19192 Signed-off-by: Peng Zhang (cherry picked from commit 2d25a9c0c760664d3de33ecca0e0272c1031cd46) commit 3cb192779a1c3662d27bba7715eb31c5f7b6a5b7 Author: Oleh Prypin Date: Thu Jun 30 00:51:50 2016 +0300 brag: Assume there are 0 MDS instead of crashing when data is missing Signed-off-by: Oleh Prypin commit 8bed107b84efdc8c735245cdfb51bfd8d07da13b Author: David Disseldorp Date: Fri Feb 10 19:19:46 2017 +0100 doc: update description of rbdmap unmap[-all] behaviour Fixes: http://tracker.ceph.com/issues/18884 Signed-off-by: David Disseldorp (cherry picked from commit f987396e126d5e61240a6645ffed439f79b072b4) commit da4e0b56c60f4bc2c67daa5dfe4d5255ab8bfc03 Author: Nathan Cutler Date: Thu Dec 15 18:23:41 2016 +0100 doc: add verbiage to rbdmap manpage Fixes: http://tracker.ceph.com/issues/18262 Signed-off-by: Nathan Cutler (cherry picked from commit fbac4a081547d83bb2436cd60b0b7ee7250f8a6c) commit 167d4fd7ccf0cdac536f95250bbfa3e9879ab769 Author: David Disseldorp Date: Fri Feb 10 17:50:12 2017 +0100 rbdmap: unmap RBDMAPFILE images unless called with unmap-all When called with a "map" parameter, the rbdmap script iterates the list of images present in RBDMAPFILE (/etc/ceph/rbdmap), and maps each entry. When called with "unmap", rbdmap currently iterates *all* mapped RBD images and unmaps each one, regardless of whether it's listed in the RBDMAPFILE or not. This commit adds functionality such that only RBD images listed in the configuration file are unmapped. This behaviour is the new default for "rbdmap unmap". A new "unmap-all" parameter is added to offer the old unmap-all-rbd-images behaviour, which is used by the systemd service. Fixes: http://tracker.ceph.com/issues/18884 Signed-off-by: David Disseldorp (cherry picked from commit e58413abf408cbe254232e563f3e30d2dc0d707c) commit 39aab763a44e45e025c311cdfff95116df11a4c4 Author: Nathan Cutler Date: Thu Dec 15 13:01:02 2016 +0100 Revert "dummy: reduce run time, run user.yaml playbook" This reverts commit d4e3cec1851ae35889127b90912e133178085bc6. Fixes: http://tracker.ceph.com/issues/18259 Signed-off-by: Nathan Cutler (cherry picked from commit a8a2a8c3e68b910dbaeb3186576898bf9f89f9fd) commit 2e50fe1684f73ebe96969c341242b6f20c8470a0 Author: Casey Bodley Date: Thu Oct 20 15:01:01 2016 -0400 rgw: fix break inside of yield in RGWFetchAllMetaCR the yield macro is implemented with for/switch, so the breaks in RGWFetchAllMetaCR weren't being applied to the for loop as expected - so any of these breaks send RGWFetchAllMetaCR into an infinite loop removed the yield {} block, so that breaks will apply to the for loop as intended, then added a single yield; statement to allow the entries_index consumer to run one per iteration Fixes: http://tracker.ceph.com/issues/17655 Signed-off-by: Casey Bodley (cherry picked from commit 190bd385a7be52867d65740c410884f5c8cbc21f) commit dc4e7a1a865ea0ae7362c1b6a7a542aa5f72107d Author: Casey Bodley Date: Fri Mar 3 12:10:40 2017 -0500 rgw: delete_system_obj() fails on empty object name Signed-off-by: Casey Bodley (cherry picked from commit 67401193f871db95a6045915fa59dce8c5dd1012) commit e9a577c8c535702d1eb285429978bdbb395e2d5c Author: Casey Bodley Date: Fri Mar 3 11:42:45 2017 -0500 rgw: if user.email is empty, dont try to delete Fixes: http://tracker.ceph.com/issues/18980 Signed-off-by: Casey Bodley (cherry picked from commit 022ecf0fcc8e44912c8758ee1d9a452dc23cbbce) commit 90de64bd81fedcb9540e40d50420e169a4a81248 Author: Alexey Sheplyakov Date: Mon Mar 20 14:05:17 2017 +0400 jewel: osd/PGLog: reindex properly on pg log split When PGLog::IndexedLog::split_into runs it builds the list, which means the old indices are wrong (point to bad memory), but index() will not rebuild them because ever since b858e86 we won't rebuild them if they are already built. Fix that by calling unindex() before the split. Based on 643ae42cf27f16dd6ed4e1402acc0483bb9fca74. Notice that both the child and the parent log are re-indexed in Jewel, so the only problem is missing unindex(). Signed-off-by: Alexey Sheplyakov commit ae498e84ffcff7424721f0d2704ec739d1cc092d Author: liuchang0812 Date: Tue Dec 20 13:21:40 2016 +0800 os/filestore: fix clang static check warn use-after-free Signed-off-by: liuchang0812 (cherry picked from commit 1d359455b3dd6abb383542ba596a03f14ac54dbd) See: http://tracker.ceph.com/issues/19311 Signed-off-by: yaoning commit 335b5fa4a9694620546422f9a02bdcc16549d7cc Author: Mykola Golub Date: Thu Feb 2 11:11:35 2017 +0100 rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly Fixes: http://tracker.ceph.com/issues/18335 Signed-off-by: Mykola Golub (cherry picked from commit 596e5ea8a5df72002672eef0a6d20572ca6f60f0) Conflicts: qa/workunits/rbd/rbd-nbd.sh: the original commit removes TOO_LARGE_IMAGE test, do the same thing src/tools/rbd_nbd/rbd-nbd.cc: help git to add "include/stringify.h" Other changes: src/tools/rbd_nbd/rbd-nbd.cc: #include so the code compiles Signed-off-by: Alexey Sheplyakov commit ced799f9c6558482d538f8dec854c62162685ad0 Author: Michal Koutný Date: Thu Jan 26 16:08:09 2017 -0500 rgw: Use decoded URI when verifying TempURL Instead of calliing url_decode directly, we reuse s->decoded_uri that is initialized in RGWREST::preprocess(). Fixes: http://tracker.ceph.com/issues/18590 Adapted from 4e1318f4dcbfd64c3ec94f4addf6e38ddd6c013a. Cherry-picking that patch requires a quite a number of unrelated changes, hence this patch does s/s->info.request_uri/s->decoded_uri/ to keep the fix as minimal as possible. Signed-off-by: Alexey Sheplyakov commit 043d70461c1eb874d9185f9bd671930fad05ff65 Author: Ilya Dryomov Date: Wed Mar 1 17:19:04 2017 +0100 osd/OSDMap: don't set weight to IN when OSD is destroyed Since commit 4e28f9e63644 ("osd/OSDMap: clear osd_info, osd_xinfo on osd deletion"), weight is set to IN when OSD is deleted. This changes the result of applying an incremental for clients, not just OSDs. Because CRUSH computations are obviously affected, pre-4e28f9e63644 servers disagree with post-4e28f9e63644 clients on object placement, resulting in misdirected requests. Fixes: http://tracker.ceph.com/issues/19119 Signed-off-by: Ilya Dryomov (cherry picked from commit a6009d1039a55e2c77f431662b3d6cc5a8e8e63f) commit 6b5322c5f62f4b90c4206c6ddcc70d090fa7eeb9 Author: Kefu Chai Date: Tue Mar 7 18:49:46 2017 +0800 osd/ReplicatedPG: try with pool's use-gmt setting if hitset archive not found due to http://tracker.ceph.com/issues/19185, a hammer OSD could store pg_hit_set_info_t with "use-gmt = true" even the pool setting is false. so we use the pool setting as a fallback if the hitset archive is not found locally and the pool.use_gmt does not match with hitset.use_gmt. Fixes: http://tracker.ceph.com/issues/19185 Signed-off-by: Kefu Chai Conflicts: osd/ReplicatedPG.cc: This cannot be cherry-picked from master because: hammer should upgrade to jewel first before moving to a higher version. so there is no necessary to include this workaround in releases later than jewel. commit d30c4d55ad52e2b63fdbd06ce256d92fc3fd36c9 Author: Abhishek Lekshmanan Date: Thu Feb 16 17:40:50 2017 +0100 doc: rgw: make a note abt system users vs normal users Mention that system users don't behave like normal users in context of normal rest operations Fixes: http://tracker.ceph.com/issues/18889 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit a47bcf70c9f51a6601b809cba219f5615b204d34) Conflicts: doc/radosgw/multisite.rst (trivial whitespace difference) commit 5ee8feaba469886f9e3bd3909475ffef62ba261d Author: root Date: Tue Feb 7 14:37:36 2017 +0530 rgw: Let the object stat command be shown in the usage Fixes: http://tracker.ceph.com/issues/19013 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 0fe76f83d19be098ef54fb0492a376fef3aa9e23) commit 754b4a482cb0369215beed58103a1e241231cf77 Author: root Date: Tue Feb 21 16:33:29 2017 +0530 rgw: Correct the return codes for the health check feature Fixes: http://tracker.ceph.com/issues/19025 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 4da2bf310f6d43423554c32e43ebf90ad2c3f3a9) commit 9cd7dd84909abdb9e603ff3aeb9958cdab8c70ad Author: Ronak Jain Date: Wed Feb 22 12:03:46 2017 +0530 rgw: Fixes typo in rgw_admin.cc Issue: http://tracker.ceph.com/issues/19026 Signed-off-by: Ronak Jain (cherry picked from commit 58837ef6ce8cbcfc2cac29d5f833b2cf62d8737a) commit 85fbb00f6ef5f11bc5d615ccd8e2202ce3896fd1 Author: Yehuda Sadeh Date: Mon Feb 27 10:35:01 2017 -0800 rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so Fixes: http://tracker.ceph.com/issues/19096 rgw_obj_key currently deals with the bucket index key, and not representing a (name, instance, ns) tupple. Need to initialize it in two steps. Signed-off-by: Yehuda Sadeh (cherry picked from commit 392c5d9dae6ba699014ffe6e1e67818fa62d7e41) commit ec0668c201a71b4a17ef0ab3c5908f57229aa6ef Author: Casey Bodley Date: Tue Nov 29 11:29:41 2016 -0500 rgw: fix for broken yields in RGWMetaSyncShardCR Fixes: http://tracker.ceph.com/issues/18076 Signed-off-by: Casey Bodley (cherry picked from commit e62d48a9bf2e309eab1a863f167af5267ebcc371) commit 6afe3efa4b636ede1cd77086cb2e70ed09fa2e95 Author: Abhishek Lekshmanan Date: Mon Jul 25 11:21:11 2016 +0200 rgw: kill a compile warning for rgw_sync killing the compile warning for ``` /ceph/src/rgw/rgw_sync.cc:1462:12: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wparentheses] if (can_adjust_marker) yield { ^ ``` Signed-off-by: Abhishek Lekshmanan Signed-off-by: Casey Bodley (cherry picked from commit 831640bb46621a6f003ad562cef7928ffa9a7ad3) commit 06916a8798439ec033294d791749ce7381d92f51 Author: Vikhyat Umrao Date: Mon Feb 13 23:07:25 2017 +0530 rgw: change log level to 20 for 'System already converted' message Fixes: http://tracker.ceph.com/issues/18919 Signed-off-by: Vikhyat Umrao (cherry picked from commit 55b567c767830170d04de4cdc8f10aba30a3f379) commit 4c1f302f7d71bedb0dead220f17eeb84e7e3f737 Author: Jing Wenjun Date: Wed Jan 11 05:28:43 2017 +0800 rgw: the swift container acl should support field .ref On the openstack-swift. The container acl supports .ref, which is ignored on ceph swift. Fixes: http://tracker.ceph.com/issues/18484 Signed-off-by: Jing Wenjun (cherry picked from commit b06f9cd9f0900db7b0d0fbcaea69cdd0d4b10132) Conflicts: src/rgw/rgw_acl_swift.cc - no extract_referer_urlspec() in jewel see https://github.com/ceph/ceph/pull/8657 commit 714eb863c30df4e653068e6ea16630504e58b704 Author: xie xingguo Date: Tue Jun 14 19:32:01 2016 +0800 server: negative error code when responding to client As the comment suggests. Also a zero or positive return code shall indicates a success, which does not match our intention here. Signed-off-by: xie xingguo (cherry picked from commit 26931f888ce4661765cca106b3a3dc66702266df) Signed-off-by: Jan Fajerski commit 1a4e1e09b1e562bf97cfe96f5cb9f937b6987165 Author: Jason Dillaman Date: Tue Dec 13 14:10:58 2016 -0500 librbd: remove image header lock assertions This assertions can sporadically fail if the watch is lost and recovered in the background. Upon a true loss of the lock, the client would either be blacklisted or it would have completed all in-flight ops before releasing. Fixes: http://tracker.ceph.com/issues/18244 Signed-off-by: Jason Dillaman (cherry picked from commit ce4f2a52ec0a794d89e7576b59c9b9aefe3db288) Conflicts: src/librbd/operation/SnapshotCreateRequest.cc: rbd class does not support the snapshot namespaces in Jewel, skip the corresponding argument commit cebba011e502f7009208bbddc295eb17f88f1bb9 Author: Dan van der Ster Date: Fri Dec 9 22:06:26 2016 +0100 os/filestore/HashIndex: be loud about splits Filestore splits are a rare yet important enough event that an OSD should visibly report when they happen. Without this reporting an operator could spend hours trying to understand the cause of any split-induced slow requests. Fixes: http://tracker.ceph.com/issues/18235 Signed-off-by: Dan van der Ster (cherry picked from commit 61c47acd3a1f3e01f0106d4a541bb7f28a1301d8) commit 1d054c3856a63ceebe44f66ff83fda691c374f71 Author: Nathan Cutler Date: Thu Mar 2 12:41:07 2017 +0100 build/ops: add psmisc dependency to ceph-base (deb and rpm) Fixes: http://tracker.ceph.com/issues/19129 Signed-off-by: Nathan Cutler (cherry picked from commit 769b695465162bc8424abf8e2f259e6765b5bbff) Conflicts: debian/control (jewel does not have f11acf2b 7e71cd2c) commit 6add2a457e2826b71c0e9e82c6f6686cecbc4584 Author: Jing Wenjun Date: Fri Nov 25 21:31:22 2016 +0800 rgw: metadata sync info should be shown at master zone of slave zonegroup When executing 'radosgw-admin sync status', the metadata sync info should be shown on the srceen at master zone of slave zonegroup. Using the function store->is_meta_master() instead of 'zonegroup.is_master && zone.id == zonegroup.master_zone' Fixes: http://tracker.ceph.com/issues/18091 Signed-off-by: Jing Wenjun (cherry picked from commit c12d0af2f98b246a73cc3ee027449a22192795b3) commit 0e11a938c5c9acd8a50efa9a154ea3bf21bcafc5 Author: Boris Ranto Date: Wed Jan 25 12:39:40 2017 +0100 systemd: Start OSDs after MONs Currently, we start/stop OSDs and MONs simultaneously. This may cause problems especially when we are shutting down the system. Once the mon goes down it causes a re-election and the MONs can miss the message from the OSD that is going down. Resolves: http://tracker.ceph.com/issues/18516 Signed-off-by: Boris Ranto (cherry picked from commit 7f4acf45dd0d86e7d9992a8c30e5876fb57b1914) Conflicts: systemd/ceph-osd@.service (jewel does not have 4179aa8d) commit 3bdd4398f1dcad0b7e22f1750ca524b97feca15a Author: yaoning Date: Mon Jun 6 13:31:52 2016 +0800 osd: preserve allocation hint attribute during recovery Signed-off-by: yaoning (cherry picked from commit e15be792960da6bac2bd469acf7d30007be61781) Conflicts: src/osd/ReplicatedBackend.cc (in master, it contains alloc_hint_flags for set_alloc_hint) src/osd/ReplicatedPG.cc (in master, it contains alloc_hint_flags in object_info_t struct) src/osd/osd_types.cc (in master, it contains alloc_hint_flags in message serialization) alloc_hint_flags is used in master bluestore, filestore does not use alloc_hint_flags. therefore, remove alloc_hint_flags here in jewel Signed-off-by: yaoning commit 8d0140a9eda814beadf1f59c9b4205f30a1d2e35 Author: Jason Dillaman Date: Tue Jan 24 09:24:52 2017 -0500 librbd: improve debug logging for lock / watch state machines Signed-off-by: Jason Dillaman (cherry picked from commit cc046597983bd491cc66081cc33d9046264fe24b) Conflicts: NOTE: cherry-picked from kraken commit to avoid conflicts commit 62ce3461c3b205eaa9062113526cf572184d0a27 Author: Jason Dillaman Date: Mon Jan 23 21:24:41 2017 -0500 test: use librados API to retrieve config params The CephContext object is not ABI-stable, so it is necessary to use the "conf_get" librados methods to safely retrieve a setting. Fixes: http://tracker.ceph.com/issues/18617 Signed-off-by: Jason Dillaman (cherry picked from commit 2ed02f3cd56bf89984c3538ac3f21ec2321cd3b7) Conflicts: src/test/librbd/test_librbd.cc (jewel does not have 006138e2d80b779d8c15b141002bb4b3852f6c4a or cb3712e08cdc2c37a983b479f4692bbdfe83b220) commit 01d04e28db7c2969b86df5b38a20b9eb156cf393 Author: Nathan Cutler Date: Thu Feb 2 23:23:54 2017 +0100 tests: Thrasher: eliminate a race between kill_osd and __init__ If Thrasher.__init__() spawns the do_thrash thread before initializing the ceph_objectstore_tool property, do_thrash races with the rest of Thrasher.__init__() and in some cases do_thrash can call kill_osd() before Trasher.__init__() progresses much further. This can lead to an exception ("AttributeError: Thrasher instance has no attribute 'ceph_objectstore_tool'") being thrown in kill_osd(). This commit eliminates the race by making sure the ceph_objectstore_tool attribute is initialized before the do_thrash thread is spawned. Fixes: http://tracker.ceph.com/issues/18799 Signed-off-by: Nathan Cutler (cherry picked from commit b519d38fb1967628ad8a1c46fcfb3f984de58790) Conflicts: qa/tasks/ceph_manager.py (jewel has only one if statement after "self.thread = gevent.spawn(self.do_thrash)" while master has four; jewel lacks 66836c957ffd974dec136997e23261ec7de2f0aa which disables ceph-objectstore-tool testing in master) commit 08a667883b68ccc72e3a4bc3013856deef1df93d Author: Nathan Cutler Date: Sat Feb 18 14:33:25 2017 +0100 rpm: build ceph-resource-agents by default To align with debian build Fixes: http://tracker.ceph.com/issues/17613 Signed-off-by: Nathan Cutler (cherry picked from commit 3e157bf16c3020ac11cb26df5df3ed331faf3c25) commit d22becab0f2c541584ce891d392760a5c4f1d153 Author: Yan Jun Date: Thu Jul 14 19:10:29 2016 +0800 msg/simple: cleanups should save the `errno` which may be changed by `ldout` and/or `<<` operator Signed-off-by: Yan Jun (cherry picked from commit 91a29bc490fdfbbef0875fa620c7ba1a1a6492ae) Signed-off-by: Robin H. Johnson commit a18a2dd108678d2e4b57e08b559c1f9a262d6923 Author: Kefu Chai Date: Fri Jun 17 13:58:55 2016 +0800 msg/simple: set close on exec on server sockets mds execv() when handling the "respawn" command, to avoid fd leakage, and enormous CLOSE_WAIT connections after respawning, we need to set FD_CLOEXEC flag for the socket fds. Fixes: http://tracker.ceph.com/issues/16390 Signed-off-by: Kefu Chai (cherry picked from commit f019ad563ce90f5aea0d8dd8b7b98688441596e0) commit 91a968b8fc7b363cae351b8648259211a1e71d18 Author: Kefu Chai Date: Fri Jun 17 01:17:05 2016 +0800 msg/async: set close on exec on server sockets mds execv() when handling the "respawn" command, to avoid fd leakage, and enormous CLOSE_WAIT connections after respawning, we need to set FD_CLOEXEC flag for the socket fds. Fixes: http://tracker.ceph.com/issues/16390 Signed-off-by: Kefu Chai (cherry picked from commit eaf68c724144d07f9506037a14d9192cb9f16d70) Conflicts: src/msg/async/AsyncMessenger.cc: Processor::accept(): applied the hunk manually (invoke set_close_on_exec on a socket returned by accept) commit 547e867628975c7144590e9332aa62b0ef82a433 Author: Jason Dillaman Date: Thu Jan 5 12:12:57 2017 -0500 librbd: possible deadlock with flush if refresh in-progress Fixes: http://tracker.ceph.com/issues/18419 Signed-off-by: Jason Dillaman (cherry picked from commit b95f92a5572d3035c20eba07e76d2c825a9853f7) Conflicts: src/librbd/ImageState.h (master commit just adds a function declaration, so just add it to jewel as well) commit 07501dec6f1c70afd4e4c2a50d7f874c39f2220b Author: Sage Weil Date: Wed Jan 18 17:02:54 2017 -0600 mon/OSDMonitor: make 'osd crush move ...' work on osds Currently it only allows you to move buckets, which is annoying and much less useful. To move an OSD you need to use create-or-move, which is harder to use. Fixes: http://tracker.ceph.com/issues/18587 Signed-off-by: Sage Weil (cherry picked from commit 47956475dea8bb8e07331dd76344a60b776b5158) Conflicts: qa/workunits/mon/crush_ops.sh: adapted "ceph osd find" to jewel syntax commit 7c6c3c753ccdd3baea834338e1a761f05b4e0a12 Author: Vikhyat Umrao Date: Thu Feb 16 23:51:11 2017 +0530 auth: 'ceph auth import -i' overwrites caps, if caps are not specified in given keyring file, should alert user and should not allow this import. Because in 'ceph auth list' we keep all the keyrings with caps and importing 'client.admin' user keyring without caps locks the cluster with error[1] because admin keyring caps are missing in 'ceph auth'. [1] Error connecting to cluster: PermissionDeniedError Fixes: http://tracker.ceph.com/issues/18932 Signed-off-by: Vikhyat Umrao (cherry picked from commit 90144aa64c11a685b6a7cb3aafea75d427f569be) commit 8c7a1df251e8289e7cf2df5b3096b91d8640695d Author: Sage Weil Date: Tue Feb 14 15:00:09 2017 -0500 osd/PG: restrict want_acting to up+acting on recovery completion On recovery completion we recalculate want_acting to see if we should add recently backfilled osds into acting. However, at this point we may have gotten infos from others OSDs outside of up/acting that could be used for want_acting. We currently assert that only up/acting osds are used in PG::RecoveryState::Active::react(const AdvMap&), so we must restrict want_acting to up/acting here. We could remove this restriction, but it would mean 1) checking on every map change that want_acting hasn't been invalidated, and if so, recalculating want_acting and requesting a new pg_temp. Also, presumably 2) on each new info, checking whether we can construct a better want_acting, and if so, doing it. That would be a good thing, but is a more complicated change. In reality this case comes up very rarely, so simply make our post-recovery want_acting calculation limit itself to up+acting. See 1db67c443d84dc5d1ff53cc820fdfd4a2128b680 for the assertion. Signed-off-by: Sage Weil (cherry picked from commit 0f2dee9aa48a00a7f2f809cd4d20e98df771da81) commit cfa37d6a1674e3f6f8eef4d8519823a7af70df01 Author: craigchi Date: Thu Feb 16 19:21:48 2017 +0800 ceph-disk: Fix getting wrong group name when --setgroup in bluestore ceph-disk prepare --setgroup will be wrong when using with bluestore Signed-off-by: craigchi (cherry picked from commit a8c0870e7370a0948e8e7fd53d3376b85bf9c649) commit 2d17092fab8080f819369d74d4c76d8ae58d899b Author: Alexey Sheplyakov Date: Tue Feb 7 16:47:45 2017 +0400 ceph-osd: --flush-journal: sporadic segfaults on exit FileStore holds a number of recources like op thread pool and work queue, key/value DB threads, etc. These should be properly stopped (released) before exiting to avoid segfaults on exit. Note: more code paths (mkfs, dump_journal, etc) need similar fixes, these will be submitted as separate patches. Fixes: http://tracker.ceph.com/issues/18820 Signed-off-by: Alexey Sheplyakov (cherry picked from commit 00184814c156f6194a6ba4b696073ca1c18a3f8f) Adjustments: - release g_ceph_context in the same way as the main code path does commit d012c381e8c59994ea9a40dc006d23f1bdd6a026 Author: Sebastien Ponce Date: Tue May 10 11:27:59 2016 +0200 radosstriper : protect aio_write API from calls with 0 bytes an assertion was failing so far, while we only have to return without doing anything Signed-off-by: Sebastien Ponce (cherry picked from commit 7cce1e8c51640f466d8bb37a21c0d5f1b00db8ab) commit 915dbace5ddea69fff29f7965f213229b6fbc0ac Author: Jason Dillaman Date: Tue Jan 17 11:55:00 2017 -0500 osdc: cache should ignore error bhs during trim A read error (such as injecting a timeout into an OSD op) might result in a bh in an error state. These should be trimable by the cache. Fixes: http://tracker.ceph.com/issues/18436 Signed-off-by: Jason Dillaman (cherry picked from commit 5910ed9de9856b5821488a1836487bbbd3d6460e) commit 419c9926d9ed57cb60228bc95956a9a1471b92cb Author: Piotr Dałek Date: Tue Jan 31 16:07:18 2017 +0100 OSD: allow client throttler to be adjusted on-fly, without restart This patch allows the osd_client_message_cap and osd_client_message_size_cap to be adjusted on-fly, using admin socket functionality. Fixes: http://tracker.ceph.com/issues/18791 Signed-off-by: Piotr Dałek (cherry picked from commit 64c309d7e18a975931b526e6f5d6f610c3a0d632) Conflicts: src/osd/OSD.cc (suppressed post-jewel option) commit 957c19b844fb44cde78ad59f872815f82bbf23b8 Author: Ali Maredia Date: Thu Nov 10 13:58:35 2016 -0500 swift: added "--cluster" to rgw-admin command for multisite support Signed-off-by: Ali Maredia commit 8423bc1eefa45366bdd215a17c61701c9b05dfdd Merge: 6dc30c4140 c078534376 Author: Andrew Schoen Date: Thu Apr 23 14:49:30 2015 -0500 Merge pull request #470 from ceph/wip-remote Add timeouts to Remote connection functions commit 6dc30c4140a833fd6cd126f8b5c1eceebad90510 Merge: d55484f9e5 c078534376 Author: Andrew Schoen Date: Mon Apr 20 12:38:29 2015 -0500 Merge pull request #466 from ceph/wip-11426 Log stderr in get_latest_image_version_deb() commit d55484f9e562779e7d47a8f63ff029337dc01eef Merge: c078534376 cd72cf2b31 Author: Dan Mick Date: Wed Apr 15 10:30:46 2015 -0700 Merge pull request #462 from ceph/wip-ssh-keys When modifying authorized_keys, store a backup commit c078534376d594aa3bf70d1d2e5dfc09ca8ae248 Merge: cd72cf2b31 fce2ed683f Author: Zack Cerza Date: Tue Apr 14 11:38:15 2015 -0600 Merge pull request #460 from zhouyuan/mkdir_p Make parent directories as needed commit cd72cf2b3132e7c6371aa4dbfe7564ad3ad0509e Author: Yehuda Sadeh Date: Mon Nov 24 09:59:30 2014 -0800 swift: set full access to subusers creation Default subuser permissions are 'none'. Signed-off-by: Yehuda Sadeh commit fce2ed683fcd3798db968a40200f8e8f215595fa Author: Zack Cerza Date: Wed Aug 6 10:06:34 2014 -0600 Remove most ceph-specific tasks. They are in ceph-qa-suite now. Signed-off-by: Zack Cerza commit fac452ae55594aea482db5c13a0bd0207b6ecff6 Author: Zack Cerza Date: Thu Mar 27 11:35:28 2014 -0500 Revert "Lines formerly of the form '(remote,) = ctx.cluster.only(role).remotes.keys()'" This reverts commit d693b3f8950ffd1f2492a4db0f8234fee31f00f0. commit e98b107302e06fa5c3c628a7ab0e7455de9ab568 Author: Warren Usui Date: Fri Feb 28 19:13:40 2014 -0800 Lines formerly of the form '(remote,) = ctx.cluster.only(role).remotes.keys()' and '(remote,) = ctx.cluster.only(role).remotes.iterkeys()' would fail with ValueError and no message if there were less than 0 or more than 1 key. Now a new function, get_single_remote_value() is called which prints out more understandable messages. Fixes: 7510 Reviewed-by: Josh Durgin Signed-off-by: Warren Usui commit e5fe884edfec66cc7e520938bd520aa8f1344f85 Merge: 8e2cdbf5ed f8bf53c4fe Author: Alfredo Deza Date: Wed Feb 19 16:40:17 2014 -0500 Merge pull request #186 from ceph/wip-7369 Fix #7369: "sed expression must be raw string" commit 8e2cdbf5ede871ebde260e6bdaec13daae03cfc5 Merge: 132f3e8ae6 f8bf53c4fe Author: Zack Cerza Date: Fri Feb 14 14:59:04 2014 -0600 Merge pull request #188 from ceph/wip-calamari-onefile Add Calamari test tasks, test script commit 132f3e8ae664a35cad89896b7300be13b5c604ec Merge: 24363351c3 7b63876676 Author: Zack Cerza Date: Fri Feb 14 11:50:55 2014 -0600 Merge pull request #192 from ceph/wip-6537-wusui Readjust the indentation of mon_clock_skew_check.py and mon_thrash.py. commit 24363351c3b1d23a8d6ef4e3f952855415f97ba8 Merge: 4eb147291f 7b63876676 Author: Zack Cerza Date: Fri Feb 14 11:45:51 2014 -0600 Merge pull request #194 from ceph/wip-6534-wusui Add docstrings to internal.py commit 4eb147291fccdb2164a46e977ee43dbf50894e5a Merge: ad9aaf8fa3 f4284b520a Author: Zack Cerza Date: Fri Feb 14 11:45:19 2014 -0600 Merge pull request #193 from ceph/wip-6538-wusui Add doc strings to Swift tests commit ad9aaf8fa35d49c33373fa69df7b38d3aca6abc5 Merge: 7b63876676 f8bf53c4fe Author: Zack Cerza Date: Fri Feb 14 11:43:15 2014 -0600 Merge pull request #187 from ceph/wip-better-debug Debug output improvements commit f4284b520a554b1cbe130731741e53a7fcf4e35f Author: Warren Usui Date: Thu Feb 13 21:11:34 2014 -0800 Add doc strings to Swift tests Fixes: 6538 Signed-off-by: Warren Usui commit 7b63876676f1a6845ba3b9147cf7bb2348ef2468 Author: Warren Usui Date: Tue Feb 11 20:21:06 2014 -0800 Add docstrings to s3 related tasks. Fixes: 6536 Signed-off-by: Warren Usui commit f8bf53c4fe52009abdf730b05e2cb2ddbb412dea Author: Zack Cerza Date: Tue Sep 24 14:19:24 2013 -0500 Fix namespace collision commit db6efe3e0ba7446ca42baf2a50eef18a10cf4a10 Merge: 611733c8b5 66555a4039 Author: Alfredo Deza Date: Tue Sep 24 08:17:22 2013 -0700 Merge pull request #106 from ceph/wip-mirror Remove lots of ceph.com hardcoding; default to upstream sources commit 66555a4039c61db9b96d6eecf8d2f298c98b6bad Author: Zack Cerza Date: Fri Sep 20 15:53:58 2013 -0500 Don't hardcode the git://ceph.com/git/ mirror Default to https://github.com/ceph/ but add a ~/teuthology.yaml option commit 611733c8b5ea55206df50c122efac612cb146c2f Merge: 2346f1d735 6e8a3807c7 Author: Sage Weil Date: Fri Sep 6 13:24:34 2013 -0700 Merge pull request #78 from ceph/wip-6247 Move helper scripts to /usr/local/bin to clean up logs. commit 6e8a3807c766f728027c3099eebfa24cdc645bd1 Author: Zack Cerza Date: Fri Sep 6 15:08:01 2013 -0500 Helper scripts live in /usr/local/bin now! commit 2346f1d735ccb40d00b8ff61d4acb446f684b3b2 Author: Joe Buck Date: Fri Aug 23 19:54:53 2013 -0700 s3tests: extend for multi-region tests Added code to the s3tests task to extract multi-region info so that that data can be added to the S3TEST_CONF file used to run S3 tests. Signed-off-by: Joe Buck Reviewed-by: Josh Durgin commit d3b6d633e161bc422f4781c7be5011f8a7ed8a32 Merge: 442a36c57c 09b01b27a3 Author: Alfredo Deza Date: Mon Aug 19 14:10:18 2013 -0700 Merge pull request #41 from ceph/wip-3791 Various usability and documentation fixes commit 442a36c57c3173b36ce17e5f85a49abf7e80ea93 Merge: 09b01b27a3 9b2c4fa4ad Author: wusui Date: Fri Aug 16 14:47:59 2013 -0700 Merge pull request #40 from ceph/wip-teutharm-wusui Wip teutharm wusui commit 09b01b27a3a1310d4257133def60896ad37fb575 Author: Zack Cerza Date: Thu Aug 15 08:49:35 2013 -0500 Fix some instances where print is being used instead of log commit 9b2c4fa4ad4c4258b26526afb0c16c71ce47f593 Author: Josh Durgin Date: Wed Jul 31 13:32:58 2013 -0700 s3/swift tests: call radosgw-admin as the right client This allows the right region and zone info to be read from ceph.conf Signed-off-by: Josh Durgin commit 3b3816df58a3ba1f2f850faf8969ad070aa0046e Author: Yehuda Sadeh Date: Thu Jul 25 16:47:34 2013 -0700 s3tests: clone correct branch Signed-off-by: Yehuda Sadeh commit f5170fb460a2183aed33122b4e6d0117a4220fec Merge: 7207a31e58 bd56af707a Author: Sandon Van Ness Date: Thu Jul 25 19:50:39 2013 -0700 Merge branch 'master' of github.com:ceph/teuthology commit 7207a31e5812dd0b29f2d6378360015622ddf4aa Merge: bd56af707a 3da945512e Author: Sandon Van Ness Date: Thu Jul 25 19:50:02 2013 -0700 Merge remote-tracking branch 'origin/wip-sandon-vm' Conflicts: teuthology/lock.py teuthology/misc.py teuthology/task/install.py commit bd56af707ae50c98ec46344cf57eb333061847b1 Merge: 3da945512e 343a42c0d8 Author: Josh Durgin Date: Fri Jul 19 14:44:51 2013 -0700 Merge branch 'wip-centos-rgw' commit 343a42c0d86af5b8630a30716c03fc84ba22f944 Author: Josh Durgin Date: Tue Jul 9 18:50:52 2013 -0700 s3tests: fix client configurations that aren't dictionaries They're always used as dictionaries later on. Signed-off-by: Josh Durgin commit 3da945512ed78081be5dbe9ba59c836a311e1973 Merge: 2c34d1971f 253cc98d98 Author: Sage Weil Date: Mon Jun 24 16:18:36 2013 -0700 Merge pull request #15 from ceph/wip-ulimits Reviewed-by: Warren Usui commit 253cc98d98855d65be7ebcdd46a39aa1004f8e67 Author: Sage Weil Date: Sun Jun 23 09:15:28 2013 -0700 enable-coredump -> adjust-ulimits and set max_files to be big, too! commit 2c34d1971f1e82311b364bf8efe60b223158d676 Merge: 61dba20d1f b366ad334a Author: Warren Usui Date: Tue May 7 19:27:51 2013 -0700 Merge branch 'wip-teuth4768a-wusui' Conflicts: teuthology/task/install.py commit 61dba20d1fce774eac5b56f0d61d4229460875c0 Merge: a9f3eb6310 5a7267f85c Author: Sage Weil Date: Mon May 6 21:31:36 2013 -0700 Merge branch 'next' commit a9f3eb631064931cbdde7ef218c16e154bdb9991 Author: Sage Weil Date: Thu May 2 13:47:46 2013 -0700 s3tests: add force-branch with higher precdence than 'branch' This way we can force a branch despite something in overrides. Signed-off-by: Sage Weil commit b366ad334af55867ba781e22c8f87b6ac7775bf2 Merge: 2a51e32891 5a7267f85c Author: Josh Durgin Date: Wed May 1 09:52:02 2013 -0700 Merge remote branch 'origin/next' commit 5a7267f85c80f88aca1b0081b07de1de3909f2e7 Author: Josh Durgin Date: Tue Apr 30 17:07:53 2013 -0700 fix some errors found by pyflakes Signed-off-by: Josh Durgin commit f866037f045887ccc5da15404935ce3361a74a08 Author: Josh Durgin Date: Tue Apr 30 13:23:22 2013 -0700 s3tests: revert useless portion of 1c50db6a4630d07e72144dafd985c397f8a42dc5 Perhaps it was attempting to debug something, but it shouldn't have been committed. Signed-off-by: Josh Durgin commit 2dcce57c00264b85cfb906223dfb89db9cc61ba5 Author: Josh Durgin Date: Tue Apr 30 16:49:04 2013 -0700 rgw tests: remove users after each test These should all be cleanup up at some point. They're almost all the same code. Signed-off-by: Josh Durgin commit 3c604251d9014dba45f3adbb2008bac16ff6346d Author: Josh Durgin Date: Tue Apr 30 16:47:34 2013 -0700 rgw tests: clean up immediately after the test There's no need for an explicit cleanup function, so move it back to where it came from (except in s3roundtrip, which did not have it). Instead, since these use a nested contextmanager, pass through and yield to the top-level run_tasks after the nested contextmanager has finished (and thus run all the cleanup steps in the subtasks for this test). Signed-off-by: Josh Durgin commit 022bd4aa42312b80917169282cbfba655bbad6f1 Author: Yehuda Sadeh Date: Tue Apr 30 07:06:03 2013 -0700 swift, s3readwrite: add missing yield Signed-off-by: Yehuda Sadeh commit 820c72b8d0177b01f69887ec64d98702db37077c Author: Yehuda Sadeh Date: Mon Apr 29 11:24:04 2013 -0700 s3tests, s3readwrite, swift: cleanup explicitly Cleaning up test dir explicitly after run, so that consecutive runs don't fail. Signed-off-by: Yehuda Sadeh commit 2a51e328913f16917a881129b475a4aeeab24ed0 Merge: cccadb9b03 617534e769 Author: Yehuda Sadeh Date: Wed Feb 20 14:10:50 2013 -0800 Merge remote-tracking branch 'origin/wip-3634' commit cccadb9b03ca4421a5fd841a61bf252c329e3649 Merge: 3eb19c8107 fa1f89478a Author: Sage Weil Date: Tue Feb 19 21:04:24 2013 -0800 Merge branch 'unstable' Conflicts: teuthology/task/workunit.py commit fa1f89478a76373cb33cf2524e2ebf68b3cd622c Author: Sander Pool Date: Wed Feb 6 19:16:52 2013 +0000 Install ceph debs and use installed debs The ceph task installs ceph using the debian packages now, and all invocations of binaries installed in {tmpdir}/binary/usr/local/bin/ are replace with the use of the binaries installed in standard locations by the debs. Author: Sander Pool Signed-off-by: Sam Lang commit 3eb19c810725b011baacdb8a6f5b172f4720a39a Author: Sam Lang Date: Wed Jan 23 14:37:39 2013 -0600 Replace /tmp/cephtest/ with configurable path Teuthology uses /tmp/cephtest/ as the scratch test directory for a run. This patch replaces /tmp/cephtest/ everywhere with a per-run directory: {basedir}/{rundir} where {basedir} is a directory configured in .teuthology.yaml (/tmp/cephtest if not specified), and {rundir} is the name of the run, as given in --name. If no name is specified, {user}-{timestamp} is used. To get the old behavior (/tmp/cephtest), set test_path: /tmp/cephtest in .teuthology.yaml. This change was modivated by #3782, which requires a test dir that survives across reboots, but also resolves #3767. Signed-off-by: Sam Lang Reviewed-by: Josh Durgin commit 2f829870e140c87b30e5b7aa3ad237a90dcb2179 Author: Yehuda Sadeh Date: Fri Dec 21 10:20:02 2012 -0800 task/swift: change upstream repository url Signed-off-by: Yehuda Sadeh commit 334d6386753510c312898552c6f92313942786ef Merge: b8e6ce4db9 26df886d82 Author: Joao Eduardo Luis Date: Thu Nov 29 00:53:59 2012 +0000 Merge branch 'wip-mon-thrasher' commit b8e6ce4db9a603ce3523b1759c65eeadee55daa7 Author: Sage Weil Date: Thu Nov 22 13:59:58 2012 -0800 s3tests: fix typo commit 26df886d825e28c25b630887b8dcc1c8c6d687d8 Author: Yehuda Sadeh Date: Mon Nov 19 16:19:06 2012 -0800 rgw-logsocket: a task to verify opslog socket works Signed-off-by: Yehuda Sadeh commit 617534e76978acb09a9f925f18bba475a65a7dd2 Author: Sage Weil Date: Mon Sep 10 11:08:57 2012 -0700 s3tests: run against arbitrary branch/sha1 of s3-tests.git commit 7d5c7ee8c6f2bfedd193a8d3b7102b4cfe0bf74f Author: Sage Weil Date: Wed Jun 6 16:00:55 2012 -0700 pull s3-tests.git using git, not http commit ce951cf4caffd5d6883cc8dcd24372bcdf03690a Author: Sage Weil Date: Sat May 5 09:30:41 2012 -0700 ceph.newdream.net -> ceph.com commit 2b879905fcfd660e242ed1a804d1c8301d17ab84 Merge: 1ac4bb10fc 1970713a2f Author: Mark Nelson Date: Wed Mar 14 15:32:23 2012 -0500 Merge branch 'master' of github.com:ceph/teuthology commit 1970713a2fc43e3afae376712356ca93a65d9e1f Author: Sage Weil Date: Fri Mar 2 10:55:19 2012 -0800 github.com/NewDreamNetwork -> github.com/ceph commit 1ac4bb10fc4b7d8d07c44b0e92b1627c721ab925 Author: Josh Durgin Date: Tue Feb 21 14:54:33 2012 -0800 Add necessary imports for s3 tasks, and keep them alphabetical. commit 92110e5a4460281139233dcea3f629d01182d398 Author: Yehuda Sadeh Date: Tue Feb 21 12:12:03 2012 -0800 rgw: access key uses url safe chars Signed-off-by: Yehuda Sadeh commit 709d9441127fec93da74c7702cafa54a47e10e8f Author: Sage Weil Date: Sun Jan 15 22:48:33 2012 -0800 use local mirrors for (most) github urls A cronjob on ceph.newdream.net updates these every 15 minutes. Sigh. commit 9598e47949ba65030c722947dc433e38875b1bd6 Author: Tommi Virtanen Date: Mon Dec 5 10:07:25 2011 -0800 Rename "testrados" and "testswift" tasks to not begin with "test". Anything "test*" looks like a unit test, and shouldn't be used for actual code. commit 6236e7db22edac7b51fc6329188b6afa74f3fc78 Author: Yehuda Sadeh Date: Thu Nov 17 16:53:21 2011 -0800 testswift: fix config commit 1dd607cabb07126769b4beb1ba6677e21c448719 Author: Yehuda Sadeh Date: Wed Nov 16 16:00:01 2011 -0800 rgw: add swift task still not completely working (for some reason it skips all the tests) commit cb425c158085568cd92c239a071e282c74eddf1a Author: Greg Farnum Date: Fri Sep 30 09:26:42 2011 -0700 s3-tests: use radosgw-admin instead of radosgw_admin Signed-off-by: Greg Farnum commit 37d7d515345ab04c333d6fada722e432e5816eb3 Author: Tommi Virtanen Date: Fri Sep 16 11:09:45 2011 -0700 s3tests: Clone repository from github. Signed-off-by: Tommi Virtanen commit 29a242d97dd4a5a9110710027cdddb244b8b0e29 Author: Tommi Virtanen Date: Tue Sep 13 14:53:02 2011 -0700 Move orchestra to teuthology.orchestra so there's just one top-level package. commit ec49a5f263f71aa473257e3fd49d86e475fe9456 Author: Tommi Virtanen Date: Fri Sep 9 13:22:03 2011 -0700 Callers of task s3tests.create_users don't need to provide dummy "fixtures" dict. commit d7d995e82b45e6077040b467c8ef9a82a573faf7 Author: Stephon Striplin Date: Tue Aug 9 13:43:46 2011 -0700 allow s3tests.create_users defaults be overridden commit 0086109767d5bfbbc370ca13d3fe91895b207821 Author: Josh Durgin Date: Thu Jul 14 16:47:29 2011 -0700 Make targets a dictionary mapping hosts to ssh host keys. commit 1b2c96416f554c0890b2690291b9a2dc8a6dc17a Author: Tommi Virtanen Date: Wed Jul 6 14:17:24 2011 -0700 Skip s3-tests marked fails_on_rgw, they will fail anyway. commit 06fb9b95e39985630d89e1635dcd12510686d9cd Author: Tommi Virtanen Date: Tue Jul 5 09:27:28 2011 -0700 The shell exits after the command, hence there is no need for pushd/popd. commit cd524a6904bf8254edc73a9148308f642638e33d Author: Josh Durgin Date: Fri Jun 24 17:09:47 2011 -0700 Add s3tests task.