Attachment 14286 Details for Bug 13499 – Patch for 4.7

[patch] Patch for 4.7

BZ13499-v4-7.patch (text/plain), 11.36 KB, created by Martin Schwenke on 2018-07-06 01:57:08 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Martin Schwenke

Created: 2018-07-06 01:57:08 UTC

Size: 11.36 KB

patch

obsolete

>From a05322d1ca87f94b4a8240ee5467e4a3ebaa0b85 Mon Sep 17 00:00:00 2001
>From: Martin Schwenke <martin@meltin.net>
>Date: Fri, 15 Jun 2018 05:51:17 +1000
>Subject: [PATCH 1/4] ctdb-tests: Add check for non-lmaster node status in
> integration tests
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499
>
>Signed-off-by: Martin Schwenke <martin@meltin.net>
>Reviewed-by: Amitay Isaacs <amitay@gmail.com>
>(cherry picked from commit 4b008556d6b1f07fd5057af845526bf941497f18)
>---
> ctdb/tests/scripts/integration.bash | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/ctdb/tests/scripts/integration.bash b/ctdb/tests/scripts/integration.bash
>index d72c471e41a..4cdbb7c806b 100644
>--- a/ctdb/tests/scripts/integration.bash
>+++ b/ctdb/tests/scripts/integration.bash
>@@ -338,6 +338,7 @@ node_has_status ()
> 	(monon)        mpat='^Monitoring mode:ACTIVE $0$$' ;;
> 	(monoff)       mpat='^Monitoring mode:DISABLED $1$$' ;;
> 	(recovered)    rpat='^Recovery mode:RECOVERY $1$$' ;;
>+	(notlmaster)   rpat="^hash:.* lmaster:${pnn}\$" ;;
> 	*)
> 	    echo "node_has_status: unknown status \"$status\""
> 	    return 1
>-- 
>2.18.0
>
>
>From 8ebbcbe9c9bab8e77eeb5414a00f2d9f3e52728d Mon Sep 17 00:00:00 2001
>From: Martin Schwenke <martin@meltin.net>
>Date: Fri, 15 Jun 2018 05:51:45 +1000
>Subject: [PATCH 2/4] ctdb-tests: Add a simple test for database traverses
>
>This tests that volatile databases traverse correctly, including the
>case where a record was updated on a non-lmaster node.
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499
>
>Signed-off-by: Martin Schwenke <martin@meltin.net>
>Reviewed-by: Amitay Isaacs <amitay@gmail.com>
>(cherry picked from commit ec72fadecd5233234947633360fe46a3a4053c07)
>---
> ctdb/tests/simple/79_volatile_db_traverse.sh | 94 ++++++++++++++++++++
> 1 file changed, 94 insertions(+)
> create mode 100755 ctdb/tests/simple/79_volatile_db_traverse.sh
>
>diff --git a/ctdb/tests/simple/79_volatile_db_traverse.sh b/ctdb/tests/simple/79_volatile_db_traverse.sh
>new file mode 100755
>index 00000000000..50732cab330
>--- /dev/null
>+++ b/ctdb/tests/simple/79_volatile_db_traverse.sh
>@@ -0,0 +1,94 @@
>+#!/bin/bash
>+
>+test_info()
>+{
>+    cat <<EOF
>+Confirm that traverses of volatile databases work as expected
>+
>+This is a very simple example.  It writes a single record, updates it
>+on another node and then confirms that the correct value is found when
>+traversing.  It then repeats this after removing the LMASTER role from
>+the node where the value is updated.
>+
>+Expected results:
>+
>+* The expected records should be found
>+
>+EOF
>+}
>+
>+. "${TEST_SCRIPTS_DIR}/integration.bash"
>+
>+ctdb_test_init "$@"
>+
>+set -e
>+
>+cluster_is_healthy
>+
>+# Reset configuration
>+ctdb_restart_when_done
>+
>+#
>+# Main test
>+#
>+TESTDB="traverse_db.tdb"
>+
>+echo "create volatile test database $TESTDB"
>+try_command_on_node 0 $CTDB attach "$TESTDB"
>+
>+echo "wipe test database $TESTDB"
>+try_command_on_node 0 $CTDB wipedb "$TESTDB"
>+
>+echo "write foo=bar0 on node 0"
>+try_command_on_node 0 $CTDB writekey "$TESTDB" "foo" "bar0"
>+
>+echo "write foo=bar1 on node 1"
>+try_command_on_node 1 $CTDB writekey "$TESTDB" "foo" "bar1"
>+
>+echo "do traverse on node 0"
>+try_command_on_node -v 0 $CTDB catdb "$TESTDB"
>+
>+echo "do traverse on node 1"
>+try_command_on_node -v 1 $CTDB catdb "$TESTDB"
>+
>+cat <<EOF
>+
>+Again, this time with lmaster role off on node 1
>+
>+EOF
>+
>+echo "wipe test database $TESTDB"
>+try_command_on_node 0 $CTDB wipedb "$TESTDB"
>+
>+echo "switching off lmaster role on node 1"
>+try_command_on_node 1 $CTDB setlmasterrole off
>+
>+try_command_on_node -v 1 $CTDB getcapabilities
>+
>+wait_until_node_has_status 1 notlmaster 10 0
>+# Wait for recovery and new VNN map to be pushed
>+#sleep_for 10
>+
>+echo "write foo=bar0 on node 0"
>+try_command_on_node 0 $CTDB writekey "$TESTDB" "foo" "bar0"
>+
>+echo "write foo=bar1 on node 1"
>+try_command_on_node 1 $CTDB writekey "$TESTDB" "foo" "bar1"
>+
>+echo "do traverse on node 0"
>+try_command_on_node -v 0 $CTDB catdb "$TESTDB"
>+
>+num=$(echo "$out" | sed -n -e 's|^Dumped $.*$ records$|\1|p')
>+if [ "$num" = 1 ] ; then
>+	echo "OK: There was 1 record"
>+else
>+	echo "BAD: There were ${num} (!= 1) records"
>+	exit 1
>+fi
>+
>+if echo "$out" | grep -q "^data(4) = \"bar1\"\$" ; then
>+	echo "OK: Data from node 1 was returned"
>+else
>+	echo "BAD: Data from node 1 was not returned"
>+	exit 1
>+fi
>-- 
>2.18.0
>
>
>From b78c0c563a667d843875797601f3dd69bc4a68c2 Mon Sep 17 00:00:00 2001
>From: Martin Schwenke <martin@meltin.net>
>Date: Fri, 15 Jun 2018 06:01:52 +1000
>Subject: [PATCH 3/4] ctdb-server: Rename CTDB_BROADCAST_VNNMAP ->
> CTDB_BROADCAST_ACTIVE
>
>This broadcast is misnamed.  Both places where this type of broadcast
>is used expect the broadcast to go to all active nodes.
>
>Make the corresponding change to the semantics in the daemon by
>sending to all active nodes.
>
>There is a mismatch between the ideas of VNN map and active nodes.  A
>node that is not in the VNN map but is active can still host database
>records.  These were the same until the LMASTER capability was
>introduced and then the logic was not updated.
>
>The only place where the VNN map is relevant is when finding the
>location master of a record in the migration code.
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499
>
>Signed-off-by: Martin Schwenke <martin@meltin.net>
>Reviewed-by: Amitay Isaacs <amitay@gmail.com>
>(cherry picked from commit 36938bfdd075a174daecb466085702adfe6a6c09)
>---
> ctdb/protocol/protocol.h       |  2 +-
> ctdb/protocol/protocol_debug.c |  4 ++--
> ctdb/server/ctdb_control.c     |  4 ++--
> ctdb/server/ctdb_ltdb_server.c | 12 +++++++++---
> ctdb/server/ctdb_server.c      | 16 ++++++++++------
> ctdb/server/ctdb_traverse.c    |  4 ++--
> 6 files changed, 26 insertions(+), 16 deletions(-)
>
>diff --git a/ctdb/protocol/protocol.h b/ctdb/protocol/protocol.h
>index e116b3a98b8..07554b585e8 100644
>--- a/ctdb/protocol/protocol.h
>+++ b/ctdb/protocol/protocol.h
>@@ -43,7 +43,7 @@ enum ctdb_operation {
> /* send a broadcast to all nodes in the cluster, active or not */
> #define CTDB_BROADCAST_ALL    0xF0000002
> /* send a broadcast to all nodes in the current vnn map */
>-#define CTDB_BROADCAST_VNNMAP 0xF0000003
>+#define CTDB_BROADCAST_ACTIVE 0xF0000003
> /* send a broadcast to all connected nodes */
> #define CTDB_BROADCAST_CONNECTED 0xF0000004
> /* send a broadcast to selected connected nodes */
>diff --git a/ctdb/protocol/protocol_debug.c b/ctdb/protocol/protocol_debug.c
>index 7c6d862a500..163bb41ba32 100644
>--- a/ctdb/protocol/protocol_debug.c
>+++ b/ctdb/protocol/protocol_debug.c
>@@ -262,8 +262,8 @@ static void ctdb_pnn_print(uint32_t pnn, FILE *fp)
> 		fprintf(fp, "CURRENT");
> 	} else if (pnn == CTDB_BROADCAST_ALL) {
> 		fprintf(fp, "ALL");
>-	} else if (pnn == CTDB_BROADCAST_VNNMAP) {
>-		fprintf(fp, "VNNMAP");
>+	} else if (pnn == CTDB_BROADCAST_ACTIVE) {
>+		fprintf(fp, "ACTIVE");
> 	} else  if (pnn == CTDB_BROADCAST_CONNECTED) {
> 		fprintf(fp, "CONNECTED");
> 	} else if (pnn == CTDB_MULTICAST) {
>diff --git a/ctdb/server/ctdb_control.c b/ctdb/server/ctdb_control.c
>index 6835ccaf063..f7a8b6b6e65 100644
>--- a/ctdb/server/ctdb_control.c
>+++ b/ctdb/server/ctdb_control.c
>@@ -859,7 +859,7 @@ int ctdb_daemon_send_control(struct ctdb_context *ctdb, uint32_t destnode,
> 		return -1;
> 	}
> 
>-	if (((destnode == CTDB_BROADCAST_VNNMAP) || 
>+	if (((destnode == CTDB_BROADCAST_ACTIVE) ||
> 	     (destnode == CTDB_BROADCAST_ALL) ||
> 	     (destnode == CTDB_BROADCAST_CONNECTED)) && 
> 	    !(flags & CTDB_CTRL_FLAG_NOREPLY)) {
>@@ -867,7 +867,7 @@ int ctdb_daemon_send_control(struct ctdb_context *ctdb, uint32_t destnode,
> 		return -1;
> 	}
> 
>-	if (destnode != CTDB_BROADCAST_VNNMAP && 
>+	if (destnode != CTDB_BROADCAST_ACTIVE &&
> 	    destnode != CTDB_BROADCAST_ALL && 
> 	    destnode != CTDB_BROADCAST_CONNECTED && 
> 	    (!ctdb_validate_pnn(ctdb, destnode) || 
>diff --git a/ctdb/server/ctdb_ltdb_server.c b/ctdb/server/ctdb_ltdb_server.c
>index c199aac2d1d..1962f854683 100644
>--- a/ctdb/server/ctdb_ltdb_server.c
>+++ b/ctdb/server/ctdb_ltdb_server.c
>@@ -1535,9 +1535,15 @@ static void ctdb_ltdb_seqnum_check(struct tevent_context *ev,
> 		TDB_DATA data;
> 		data.dptr = (uint8_t *)&ctdb_db->db_id;
> 		data.dsize = sizeof(uint32_t);
>-		ctdb_daemon_send_control(ctdb, CTDB_BROADCAST_VNNMAP, 0,
>-					 CTDB_CONTROL_UPDATE_SEQNUM, 0, CTDB_CTRL_FLAG_NOREPLY,
>-					 data, NULL, NULL);		
>+		ctdb_daemon_send_control(ctdb,
>+					 CTDB_BROADCAST_ACTIVE,
>+					 0,
>+					 CTDB_CONTROL_UPDATE_SEQNUM,
>+					 0,
>+					 CTDB_CTRL_FLAG_NOREPLY,
>+					 data,
>+					 NULL,
>+					 NULL);
> 	}
> 	ctdb_db->seqnum = new_seqnum;
> 
>diff --git a/ctdb/server/ctdb_server.c b/ctdb/server/ctdb_server.c
>index 900674132ac..93256ecfd58 100644
>--- a/ctdb/server/ctdb_server.c
>+++ b/ctdb/server/ctdb_server.c
>@@ -389,14 +389,18 @@ static void ctdb_broadcast_packet_all(struct ctdb_context *ctdb,
> }
> 
> /*
>-  broadcast a packet to all nodes in the current vnnmap
>+  broadcast a packet to all active nodes
> */
>-static void ctdb_broadcast_packet_vnnmap(struct ctdb_context *ctdb, 
>+static void ctdb_broadcast_packet_active(struct ctdb_context *ctdb,
> 					 struct ctdb_req_header *hdr)
> {
> 	int i;
>-	for (i=0;i<ctdb->vnn_map->size;i++) {
>-		hdr->destnode = ctdb->vnn_map->map[i];
>+	for (i = 0; i < ctdb->num_nodes; i++) {
>+		if (ctdb->nodes[i]->flags & NODE_FLAGS_INACTIVE) {
>+			continue;
>+		}
>+
>+		hdr->destnode = ctdb->nodes[i]->pnn;
> 		ctdb_queue_packet(ctdb, hdr);
> 	}
> }
>@@ -430,8 +434,8 @@ void ctdb_queue_packet(struct ctdb_context *ctdb, struct ctdb_req_header *hdr)
> 	case CTDB_BROADCAST_ALL:
> 		ctdb_broadcast_packet_all(ctdb, hdr);
> 		return;
>-	case CTDB_BROADCAST_VNNMAP:
>-		ctdb_broadcast_packet_vnnmap(ctdb, hdr);
>+	case CTDB_BROADCAST_ACTIVE:
>+		ctdb_broadcast_packet_active(ctdb, hdr);
> 		return;
> 	case CTDB_BROADCAST_CONNECTED:
> 		ctdb_broadcast_packet_connected(ctdb, hdr);
>diff --git a/ctdb/server/ctdb_traverse.c b/ctdb/server/ctdb_traverse.c
>index 04a41138a72..5ea19709599 100644
>--- a/ctdb/server/ctdb_traverse.c
>+++ b/ctdb/server/ctdb_traverse.c
>@@ -387,8 +387,8 @@ static struct ctdb_traverse_all_handle *ctdb_daemon_traverse_all(struct ctdb_db_
> 	}
> 
> 	if (ctdb_db_volatile(ctdb_db)) {
>-		/* normal database, traverse all nodes */	  
>-		destination = CTDB_BROADCAST_VNNMAP;
>+		/* volatile database, traverse all active nodes */
>+		destination = CTDB_BROADCAST_ACTIVE;
> 	} else {
> 		int i;
> 		/* persistent database, traverse one node, preferably
>-- 
>2.18.0
>
>
>From 57ccb5978a129d659c57207873586a52f6787532 Mon Sep 17 00:00:00 2001
>From: Martin Schwenke <martin@meltin.net>
>Date: Fri, 15 Jun 2018 06:07:54 +1000
>Subject: [PATCH 4/4] ctdb-docs: Fix the documentation for VNN map
>
>It is incorrectly says that nodes not in the VNN map can not be
>DMASTER.
>
>BUG: https://bugzilla.samba.org/show_bug.cgi?id=13499
>
>Signed-off-by: Martin Schwenke <martin@meltin.net>
>Reviewed-by: Amitay Isaacs <amitay@gmail.com>
>(cherry picked from commit af446d5209e37a38363911e5f339869b73d87963)
>---
> ctdb/doc/ctdb.1.xml | 7 +++----
> 1 file changed, 3 insertions(+), 4 deletions(-)
>
>diff --git a/ctdb/doc/ctdb.1.xml b/ctdb/doc/ctdb.1.xml
>index 471d8255bce..ae3b9d3b699 100644
>--- a/ctdb/doc/ctdb.1.xml
>+++ b/ctdb/doc/ctdb.1.xml
>@@ -292,10 +292,9 @@
> 	<title>Virtual Node Number (VNN) map</title>
> 	<para>
> 	  Consists of the number of virtual nodes and mapping from
>-	  virtual node numbers to physical node numbers.  Virtual
>-	  nodes host CTDB databases.  Only nodes that are
>-	  participating in the VNN map can become lmaster or dmaster
>-	  for database records.
>+	  virtual node numbers to physical node numbers.  Only nodes
>+	  that are participating in the VNN map can become lmaster for
>+	  database records.
> 	</para>
>       </refsect3>
> 
>-- 
>2.18.0
>

Flags:

amitay: review+

Actions: View

Attachments on bug 13499: 14286 | 14287 | 14307