[Crowbar] crowbar <service> proposal create racey / chef index not populated fast enough

Ralf Haferkamp rhafer at suse.de
Mon Jul 23 06:22:17 CDT 2012


Hi,

any comments on the expander patch I sent recently? Do you think it makes 
sense? Is it dangerous? I would really like to get some feedback on it.

regards,
	Ralf

On Freitag 13 Juli 2012 15:37:45 Ralf Haferkamp wrote:
> Hi,
> 
> find a very simple one attached. The patch causes chef-expander to tell
> solr to commit index immediately after every update. As I already
> mentioned I am unsure what impact this will have on chef's and solr's
> overall performance.
> 
> Another possiblity is to tweak solr's autocommit settings in
> solrconfig.xml. The current default is:
>     <autoCommit>
>       <maxDocs>100</maxDocs>
>       <maxTime>10000</maxTime>
>     </autoCommit>
> 
> Reducing those numbers should decrease the latency as well.
> 
> regards,
> 	Ralf
> 
> On Freitag 13 Juli 2012 08:16:19 Victor_Lowther at Dell.com wrote:
> > I would be interested in patches that decrease the latency of
> > chef-solr and chef-expander -- their lag is a source of lots of
> > anguish and annoying workarounds in our current codebase.
> > 
> > -----Original Message-----
> > From: crowbar-bounces On Behalf Of Ralf Haferkamp
> > Sent: Friday, July 13, 2012 7:13 AM
> > To: crowbar
> > Subject: [Crowbar] crowbar <service> proposal create racey / chef
> > index not populated fast enough
> > 
> > Hi,
> > 
> > we recently ran into trouble when creating multiple proposals on the
> > command line. Doing
> > 
> > crowbar "mysql" proposal create default; (in our setup is "database"
> > instead for "mysql" but that doesn't make a difference)
> > and
> > 
> > crowbar "keystone" proposal create default;
> > 
> > directly after another on the commandline the keystone proposal will
> > not find the mysql proposal and fallback to sqlite instead. I figured
> > out that this is because the chef search for mysql proposals that
> > KeystoneService does in create_proposal() does not return anything
> > because the updates to solr's search indexes that creating the
> > "mysql"
> > proposal would cause, weren't committed yet. This seems to be because
> > chef-expander doesn't explicitly commit it's changes to solr, but
> > relies solr's autocommit settings, currently solr is configured to
> > leave changes uncommitted for at most 10 seconds, this gives a pretty
> > big window for failures in crowbar. And I was even able to trigger
> > the
> > problem by clicking fast enough in the WebUI :).
> > 
> > There are different solutions to this:
> > - insert sleeps everywhere in script calling the crowbar cli. Or
> > alternatively to manual knife searches until the desired chef object
> > show up in the results (I would like to avoid this kind of hacks) -
> > lower the autocommit settings in the solr configuration to do commits
> > much faster - patch chef-expander to commit updates immediatly, the
> > patch for that is very simple, but I don't know what negative
> > consequence this might have for solr, and the IO system of the admin
> > node :)
> > 
> > regards,
> > 
> > 	Ralf



More information about the Crowbar mailing list