Bug 8527 - db_ctdb_traverse fails to traverse records created within the current transaction
Summary: db_ctdb_traverse fails to traverse records created within the current transac...
Status: RESOLVED FIXED
Alias: None
Product: Samba 3.6
Classification: Unclassified
Component: Clustering (show other bugs)
Version: 3.6.0
Hardware: All All
: P5 normal
Target Milestone: ---
Assignee: Karolin Seeger
QA Contact: Samba QA Contact
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-10-14 12:40 UTC by Gregor Beck (550 Unknown user)
Modified: 2012-02-18 18:45 UTC (History)
3 users (show)

See Also:


Attachments
patch for v3-6-test (2.35 KB, patch)
2011-10-14 13:40 UTC, Gregor Beck (550 Unknown user)
metze: review-
Details
Patches for v3-6-test (8.33 KB, patch)
2011-10-15 08:47 UTC, Stefan Metzmacher
obnox: review+
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gregor Beck (550 Unknown user) 2011-10-14 12:40:33 UTC
This manifests itself in a corrupted registry.tdb after uprading from registry.tdb format version 1 to 3
Comment 1 Gregor Beck (550 Unknown user) 2011-10-14 13:40:12 UTC
Created attachment 6999 [details]
patch for v3-6-test
Comment 2 Michael Adam 2011-10-14 13:43:47 UTC
Comment on attachment 6999 [details]
patch for v3-6-test

ack I did already review and test this.

Metze please also review.
Comment 3 Michael Adam 2011-10-14 14:11:34 UTC
sorry, took back ack for now: this needs some update.
there is something wrong here: the count is not handled correctly.
And we should take and update from the master to the dbwrap_traverse
wrapper that adds the count as an additional argument
Comment 4 Stefan Metzmacher 2011-10-15 08:46:17 UTC
Comment on attachment 6999 [details]
patch for v3-6-test

Incomplete
Comment 5 Stefan Metzmacher 2011-10-15 08:47:37 UTC
Created attachment 7003 [details]
Patches for v3-6-test

Michael, please check if this compiles
Comment 6 Michael Adam 2011-10-17 10:37:07 UTC
Comment on attachment 7003 [details]
Patches for v3-6-test

Metze: It compiles. :-)
We also discussed that this is the right thing to do.

ACK
Comment 7 Michael Adam 2011-10-17 10:40:41 UTC
Assigning to Karolin for inclusion into 3.6.X, whatever is fit.

Note that this but can actually be pretty bad for people upgrading from earlier samba versions to 3.6, if they store data in registy, like configuration, since it will create a registry that is partly in old and partly in new format. As a workaround, the new "net registry check" tool can be used to repair the registry, but this is only in master since we unfortunately did not finish it early enough to get it into 3.6.

Cheers - Michael
Comment 8 Karolin Seeger 2011-10-17 18:38:25 UTC
(In reply to comment #7)
> Assigning to Karolin for inclusion into 3.6.X, whatever is fit.
> 
> Note that this but can actually be pretty bad for people upgrading from earlier
> samba versions to 3.6, if they store data in registy, like configuration, since
> it will create a registry that is partly in old and partly in new format. As a
> workaround, the new "net registry check" tool can be used to repair the
> registry, but this is only in master since we unfortunately did not finish it
> early enough to get it into 3.6.
> 
> Cheers - Michael

It's too late for 3.6.1, will be included in 3.6.2.

I think we should mention that in the release notes. Do you agree?
Maybe we should add the "net registry check" tool also.
Is that cherry picking only?
Comment 9 Michael Adam 2011-10-17 21:46:25 UTC
(In reply to comment #8)
> (In reply to comment #7)
> > Assigning to Karolin for inclusion into 3.6.X, whatever is fit.
> > 
> > Note that this but can actually be pretty bad for people upgrading from earlier
> > samba versions to 3.6, if they store data in registy, like configuration, since
> > it will create a registry that is partly in old and partly in new format. As a
> > workaround, the new "net registry check" tool can be used to repair the
> > registry, but this is only in master since we unfortunately did not finish it
> > early enough to get it into 3.6.
> > 
> > Cheers - Michael
> 
> It's too late for 3.6.1, will be included in 3.6.2.

That's what I assumed.

> I think we should mention that in the release notes. Do you agree?

Yes, definitely. You mean the release notes for 3.6.1?
I can write something up if you like.

> Maybe we should add the "net registry check" tool also.

That would really be awesome.
It has already proven very useful.

> Is that cherry picking only?

It should be. We also aready have a version in
the v3-6-ctdb branch that we cherry-picked the
master copy from. It will require a couple of 
preliminary patches, I think. I can check and
propose a minimal patchset.

Cheers - Michael
Comment 10 Karolin Seeger 2011-10-18 18:03:44 UTC
Hi Micha,

(In reply to comment #9)
> (In reply to comment #8)
> > (In reply to comment #7)
> > > Assigning to Karolin for inclusion into 3.6.X, whatever is fit.
> > > 
> > > Note that this but can actually be pretty bad for people upgrading from earlier
> > > samba versions to 3.6, if they store data in registy, like configuration, since
> > > it will create a registry that is partly in old and partly in new format. As a
> > > workaround, the new "net registry check" tool can be used to repair the
> > > registry, but this is only in master since we unfortunately did not finish it
> > > early enough to get it into 3.6.
> > > 
> > > Cheers - Michael
> > 
> > It's too late for 3.6.1, will be included in 3.6.2.
> 
> That's what I assumed.
> 
> > I think we should mention that in the release notes. Do you agree?
> 
> Yes, definitely. You mean the release notes for 3.6.1?

No, I meant for 3.6.2.
If you think we should add it to the 3.6.1 release notes also, we could do that of course.

> I can write something up if you like.

Yes, please.

> > Maybe we should add the "net registry check" tool also.
> 
> That would really be awesome.
> It has already proven very useful.
> 
> > Is that cherry picking only?
> 
> It should be. We also aready have a version in
> the v3-6-ctdb branch that we cherry-picked the
> master copy from. It will require a couple of 
> preliminary patches, I think. I can check and
> propose a minimal patchset.

Ok, thanks.

Re-assigning to you as a reminder.

Cheers,
Karo
Comment 11 Stefan Metzmacher 2011-11-10 20:05:58 UTC
What's the status with attached patches? They're not in v3-6-test yet.
Comment 12 Stefan Metzmacher 2011-11-10 20:10:50 UTC
Michael, if you want other patches, please remove the review+.
Comment 13 Stefan Metzmacher 2012-01-02 08:48:00 UTC
Michael, any updates on this?
Comment 14 Michael Adam 2012-02-02 23:50:37 UTC
Gosh, this is still open!

We need this in the next release.

And the the fix for the actual traverse bug that caused the
registry corruption in the upgrade is here.

But we should definitely offer "net registry check" to be able
to repair the registry that might have gotten corrupted in an
upgrade to 3.6.X prior to these fixes.

Working on providing the patch to add "net registry check".

Michael
Comment 15 Michael Adam 2012-02-12 20:25:50 UTC
Ouch. I just realized that this is 

a) more embarassing but
b) less severe than I thought:

We do actually have registry version 2 in 3.6 and not registry version 3.
But the upgrade code is only affected by this bug if going from version
1 to version 3, because in this case two traverses (changing keys) are
run inside one transaction. The source of confusion must have been
that the v3-6-ctdb branch sports version 3...

So the good news is that we should not have broken registries by
the upgrade from 3.(<6) to 3.6.

The bad news is that we somehow have missed to get registry
version 3 into samba 3.6. :-(

So for now, I will just give the bug to Karolin in order to pick the
traverse fixes for the next 3.6 release.

And then we can later (in a separate bug, maybe) think about whether we should
move to registry version 3 later and then also provide the "net registry check"
tool along with it.

Cheers - Michael
Comment 16 Karolin Seeger 2012-02-18 18:45:34 UTC
Pushed to v3-6-test.
Closing out bug report.

Thanks!