miniBB ® miniBB®
miniBB Support Forums
 | Forums | Register | Reply | Search | Statistics | Manual |
Specific miniBB Support Forums / Specific /   

Any existing import tools?

Author sandoo
Registered
#1 | Posted: 2 Dec 2013 16:37 
I'm new to minibb.
I want a forum I can place inside a block element inside an enclosing content management system (read post reply anyway) and minibb seems to work well that way (strip off HTML HEAD and BODY) from the templates. Misc hacks and adjustments will surely arise.

My first step will be to import 20,000 posts from a Phorum forum I've been running since 2000 or so. Last 10 - 13 years. I've already written an importer that (sort of) works. But I am getting "redirect cycle" error messages in the newly imported data. I'll get it figured out eventually.

Are there any existing importers to use as a model to work with? Perhaps for importing data from phpBB?

==============

Also, I see I have write privileges to some minibb dot com forums and not to others. I do not have write permission to the developers forum. Is that something I have to "apply" for?

Sandoo (Colin Pittendrigh)

Author sandoo
Registered
#2 | Posted: 2 Dec 2013 16:39 
I haven't purchased any addons yet. But I will want quite a few of them. If I can load my existing data first.

Author Jaime
Registered
#3 | Posted: 2 Dec 2013 21:52 
Hi Sandoo and welcome here in the forum,

Are there any existing importers to use as a model to work with? Perhaps for importing data from phpBB?

Youre not the first looks for such import possibilities. Read About Converting / Migrating miniBB to/from other board scripts, Conversion: phpbb -> minibb and Import VBulletin ... No, there is not a ready tool for the job. Each forum software is different and it can therefore also be no simple import option!. But this does not mean that it would be impossible ... Ask the Premium Support :)

Author Paul
Lead Developer
#4 | Posted: 3 Dec 2013 16:29 | Edited by: Paul 
Thanks Jaime, you've said almost everything on this topic above.

sandoo:
There are so many forum software in this world, and each version may have so different structure, that's why I have sad news for you: miniBB doesn't have migration tools. Sorry. It's just impossible for the small team like we have to support thousands of softwares at the time we don't even have enough of powers sometimes to support our own.

We could program one for the specific database only within our paid support terms only, because it may appear a very specific and customized task - to perform an ideal conversion, often there is not enough a "tool for all".

sandoo:
I see I have write privileges to some minibb dot com forums and not to others. I do not have write permission to the developers forum. Is that something I have to "apply" for?

Not sure what kind of "Developers" forum do you mean? We don't have such by the title. We only have some forums which are opened only to make new messages, not new topics. Everyone incl. guests can post messages in all of our forums.

Author sandoo
Registered
#5 | Posted: 3 Dec 2013 16:46 
I am working on a migration tool for phorum. It will take a while. But I will get it done.

Author sandoo
Registered
#6 | Posted: 3 Dec 2013 20:11 | Edited by: sandoo 
OK. I have an importer for phorum that works for importing users, topics and posts.
I'll make it available at some point. Perhaps early next year.

I need to purchase some minibb addons first (images etc) so I can deal with images embedded as blobs in phorum posts. I'll have to see how the minibb file and image upload stuff works. Does minibb put images into the minibb schema as a blob? Or does it make href links to uploaded files on the server file system?

I'll figure that part out when I digest the guts of the minibb embedded images stuff. I only need to import users, posts and embedded images. I plan to forget about avatars and private messages. I have 20,000 phorum posts over ten years, and only a handful of private messages during that entire period.

Zero avatars, even though it's possible to make them in phorum.

Author Paul
Lead Developer
#7 | Posted: 3 Dec 2013 21:16 | Edited by: Paul 
sandoo:
Does minibb put images into the minibb schema as a blob? Or does it make href links to uploaded files on the server file system?

miniBB doesn't use database for storing the files, we're considering this as a very outdated and not practical approach; it stores all files as physical files on the disk. Extension might be changed, but it's not mandatory - the original extension could be kept. File/Images Attachments is purely flat-files based add-on, it doesn't use database even for storing the files information. Yes, miniBB creates either static either dynamic links to files, it depends on the privacy setting.

sandoo:
Zero avatars, even though it's possible to make them in phorum.

As you see, we don't even have smilies installed on the official forum :)
I think, whatever people need from the forum, is its content, the questions/answers model, SEO weapon for quick and easy earning.
Those nifty graphical things are mostly for women, so if your forum is die-hard-men oriented, I could merely understand why there were no avatars involved.

Good luck. You may use our File Bank (FIle Bank Icon icon above the form) to upload .zip packages and refer to them in Custom Tutorials and Modifications.
Thanks in advance.

Author sandoo
Registered
#8 | Posted: 3 Dec 2013 21:21 
Thank you for the background information. I do not like images stored as blobs inside any schema. It gains little and makes the images useless to other applications....or too difficult to use anyway.

Proponents of image blobs would say files on the file system too often get moved, resulting in dead links inside referring HTML fragments. But that's an empty argument. Broken links can easily be found and fixed. Even from cron

Author Paul
Lead Developer
#9 | Posted: 3 Dec 2013 21:56 
sandoo:
Broken links can easily be found and fixed.

Even better approach is to handle moving of files properly in the code itself. So it's surely not the argument for storing images in blob fields - most likely looks like an argument for masking the bad code.

I hope miniBB doesn't have this problem, as this add-on is compatible with moving/merging add-ons, deleting operation; and for possibly "missed" files (which could happen on a busy system with hundred thousands of topics/posts) there is also included the tool to fix the spare files left on the disk, which could be added to the cronjob to run monthly. Possibly it's still a very rare case, and useful only for keeping some bit of Mbytes on your disk.

Not sure how it would affect the system if the files are even moved internally between sectors on the disk. The system should keep all references to them, else it's a badly coded system. Probably this argument is from 80-90s when systems were not enough powerful to handle lots of inodes within one folder; but in miniBB, there could be used a folders structure when each folder refers to each topic, so there could be kept quite a lot of inodes for one forum.

miniBB archiving feature (rare but possible) also could split the large amount of folders into smaller related pieces, so most likely keeping LOTS of files is still not a problem for our product; I know forums with huge heaps of files uploaded each day, they all work stable - not sure how it would be possible to achieve if all that stuff would be kept in mySQL - this would make the database Terabytes in size! What kind of speed then could we expect from it...

Author sandoo
Registered
#10 | Posted: 3 Dec 2013 22:27 
I think we're saying the same thing, albeit in different ways.

I worked 5 years on a Java front end to an XML database (Exist and Sleepycat) for use a "laboratory database." We discovered casual users make the wrong choices more often than not. If they (any keyboard users at all) had permission to rename something manually they would do so AND they would figure out how to do it the wrong way. Reliably so. HTML fragments containing static links to the file system will get broken sooner rather than later. But that doesn't mean they can't be fixed.

We used maintenance codes that run from cron to keep things in synch. We were able to use a combination of Lucene, locate and bash/perl/sed to fix most broken links. Links that remained broken for a second consecutive 24 hour period ended up as an email to the database administrator. Relying on everyday users (like me) to do the right thing is like asking chimpanzees not to steal food. :=))

Specific miniBB Support Forums / Specific / Any existing import tools? Top

Your Reply Click this icon to move up to the quoted message

 Short link for this topic:

 ?
Only registered users are allowed to post here. Please, enter your username/password details upon posting a message, or register first.


Before posting, make sure your message is compliant with our forum posting rules. If not, it may be locked or deleted with no explanation.
 
miniBB Support Forums Powered by Forum Software miniBB ® Features  Requirements  Demo  Download  Showcase  Gallery of Arts
Compiler  Premium Extensions  Premium Support  License  Contacts
Get the Captcha add-on: protect your miniBB-forums from the automated spam and flood.
Captcha Addon for miniBB