Performance importing pages

[matthew warren]

Hallo, MoinMoin'ers

I have a very simple piece of code to import a set of articles into a
MoinMoin Wiki.

It goes like this;

<pre>
for PageName in UniqHeadings:
request = RequestCLI()
editor = PageEditor(request,PageName)
request.form={}
editor.saveText(editor.normalizeText(Articles[PageName]), 0)
</pre>

this has a list of around 4,500 UniqHeadings to get through. Average article
size is 2k. An Apache served MoinMoin instance is running.

It is currently writing pages at the rate of around 8 per minute. Do I
really have to wait 9 hours, or is something amiss?

Thanks,

Matt

_________________________________________________________________
Be the first to hear what's new at MSN - sign up to our free newsletters!
http://www.msn.co.uk/newsletters



-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005

[Nir Soffer]

Post by matthew warren
I have a very simple piece of code to import a set of articles into a
MoinMoin Wiki.
It goes like this;
<pre>
request = RequestCLI()
editor = PageEditor(request,PageName)
request.form={}
editor.saveText(editor.normalizeText(Articles[PageName]), 0)
</pre>
this has a list of around 4,500 UniqHeadings to get through. Average
article
size is 2k. An Apache served MoinMoin instance is running.
It is currently writing pages at the rate of around 8 per minute. Do I
really have to wait 9 hours, or is something amiss?

I think you miss something - looks like this code will not work - how
the code knows where is your wiki data dir with no url?

I tested this code on my idle machine (G5 Dual 2G):

import sys
sys.path = [# The path to the wiki directory
'/Volumes/Home/nir/wiki/fix',
# The path to moinmoin, not needed if its installed with
setup.py
'/Volumes/Home/nir/Projects/moin/fix'] + sys.path


from MoinMoin.PageEditor import PageEditor
from MoinMoin.request import RequestCLI


def save(url, pagename, text):
request = RequestCLI(url=url, pagename=pagename)
editor = PageEditor(request, pagename)
text = editor.normalizeText(text)
dummy, revision, exists = editor.get_rev()
return editor.saveText(text, revision)


import os
url = 'localhost/fix/'
dir = 'imports'
files = [name for name in os.listdir(dir)
if not name.startswith('.')]
for name in files:
text = file(os.path.join(dir, name)).read()
save(url, name, text)


I imported 80 2132 bytes pages in 10.4s.


Best Regards,

Nir Soffer



-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005