Exporting MediaWiki sites to Google sites

In the last post I’ve talked about moving files from SharePoint to Google Site,
what about a Mediawiki site?
how to export it and upload it?
I’ve already covered the uploading files to Google sites via the Google sites API,
you can use java or python to write a script to mass upload all your files and htmls to your new google site,
but first you need to export everything out of the Mediawiki site.

here is a working method to export your site, these are the steps:

1. install python
2. download the mw2html script
3. export the mediawiki site

so lets start :

Installing python

If you are working from a linux distro with yum its as easy as typing:
yum install python

for windows – you will need to download and install from the python site:
the main python download page for windows
or you can just grab the latest version for September 4 2011 Python 3.2.2
Download and install the msi package.

next step is to add the installation to the computer’s path,
in windows 7  open the start panel and type in the search panel:
“view advance system settings”

press the “Environment Variables”

in the System Variables section search for the “Path” line and choose edit:

now to the end of the line add the path of where you have installed the python,
for example I have installed it to  C:\Python27 , and so i will add at the end of the Path line this:
;C:\Python27
notice the “;”  - this separates between the items on the Path line.

Now if you will type “python -V” on your command line it will run the python binary and will show you your version

now that we have the python working lets move to the next step:

Download the mw2html script

you can find it here:

from Connelly Barnes blog

this script will crawl all over your media wiki site and will grab all the html files and all the attachments there.

Exporting the mediawiki site

usage:
url – URL of mediawiki page to convert to static HTML.
outdir – Output directory.

-f, –force – Overwrite existing files in outdir.
–no-flatten – Do not flatten directory structure.
–no-lower – Retain original case for output filenames and dirs.
–no-clean – Do not clean up filenames (clean replaces
non-alphanumeric chars with _, renames math thumbs).
–no-hack-skin – Do not modify skin CSS and HTML for looks.
–no-made-by – Suppress “generated by” comment in HTML source.
–no-move-href – Disable <movehref> tag. [1]
–no-remove-png – Retain external link PNG icons.
–no-remove-history – Retain image history and links to information.
-l, –left=a.html – Paste HTML fragment file into left sidebar.
-t, –top=a.html – Paste HTML fragment file into top horiz bar.
-b, –bottom=a.html – Paste HTML fragment file into footer horiz bar.
-i, –index=filename – Move given filename in outdir to index.html.

Example Usage:
mw2html http://127.0.0.1/mywiki/ out -f -i main_page.html -l sidebar.html

 Important note:

this script will crawl all over your wiki site, but it does ignore all the files that do not have a link to them,
all the  ”orphan” pages, to solve this issue, instead of pointing to the main page, you can run the mw2html script while pointing to the orphans page,
it will get all the regular files from your site and the Orphan pages.
the link looks like:

http://yourwikisite/wiki/index.php?title=Special:Lonelypages&limit=500&offset=0

now that you have a folder with all your wiki page, you can edit the content of the folder,
remove all the files staring with the word “image_”
remove all the un-needed js files,
and do internal changes in the html files like change the links pointing to the original wiki site and point them to the new Google site URL.

now you can use the Google sites API to upload all your html to your new site.

Technorati Tags: export, Google sites, Google sites API, mw2html, wikimedia

SharePoint to Google sites – how to move your files

Many organisations now move to use the Google office suite for their day-to-day office needs,
this includes the Google sites service which allows you to easily create internal portals for the different
organisation’s departments.

This move allows for a department to share an internal web site that has easy access from home as well with the regular office google account,
it also allows for better collaboration with other branches of the same company located far away – as it saves the need for an office VPN, or the need to connect to a server that is located in a branch in another country,
instead – everybody just connect to Google.

But what do we do if our files are already on the office internal SharePoint?

apparently – the migration processes from SharePoint to Google sites is not as hard ,
here are two working options.

Exporting from SharePoint

Connect to the SharePoint and browse to where you keep the files you wish to move,
choose  ”Actions –>  Open with windows explorer”

you will get and explorer folder listing all the shared files there,
at this point you can select all the folders  and copy them locally to your desktop or to a network share – if the size of the files is too large.

The only folder not to copy over is the “forms” folder – as it is a system folder and you do not need to move it too.

Uploading files to Google Sites

Here we have 2 options:

1. Use native Google Sites files upload option

   This option works well only if you have a very small number of files and folder to upload
   

    

2. Use Google Sites API

    The Google sites API  will allow you to upload multiple files and folders, 
   it support now two languages -

• Java
• Pythonand you can read some more about in the Official Google Enterprise Blog .
  The API allows you to:
• create new sites
• retrieve , create , change and delete pages
• upload and download attachments and files
• change sharing settings
• and manage templates

Technorati Tags: Google sites, Google sites API, SharePoint

Adding JMX support to your java app

Lets start with the basics -

what is JMX?

JMX is a technology that lets you add management interfaces for Java applications

the jmx interface on your java application will let you monitor and publish graphs for:

• heap memory
• threads and their stack trace
• CPU
• classes
• memory pools

Enabling JMX in Tomcat

wrapper.java.additional.1=-Dcom.sun.management.jmxremote

wrapper.java.additional.2=-Dcom.sun.management.jmxremote.port=<port>

wrapper.java.additional.3=-Dcom.sun.management.jmxremote.ssl=false

wrapper.java.additional.4=-Dcom.sun.management.jmxremote.authenticate=false

wrapper.java.additional.5=-Djava.rmi.server.hostname=<ServerName>

Things to verify before usage:

1. the numbers after the “wrapper.java.additional.” part are in increasing order – make sure to continue from where the last config line showed, and make sure there are no gaps in the counting.
2. change <port> to the port number you wish to expose
3. these settings will enable password-less connection to you JMX server, if you wish to add authentication and SSL you will need to change these settings.
4.  change <ServerName> to your server name – this setting is needed – without it the server will only allow localhost connections and wont allow remote.

  

Technorati Tags: java, jmx, tomcat

World Statistics

This is going to be a praise post for gapminder.com

I’m going over the data at the site and its amazing stuff,
the main page has lots of statistics which is shown in a new way,
the gapminder app shows the data at what i think is a 4 dimension way,
it shows x and y axis stats, and another axis represented as the size of the balls in the demo,
and how they grow over the 4th dimension – time.

Amazing stuff.

Gapminder

This is the main stats app:

you can see the advance of the bubbles over the x and y axis,
the z axis is represented with the balls size,
and its all advancing in time.
this default presentation shows Life expectancy over income per person, over population size.

wonderful demo.

Marriage Statistics

This great demo shows you the connection between women age at marriage and the amount of kids they have,
on of the things this tool does best is break myths.
I was under the impression that in Europe – Sweden for example, the marriage age went down,
and with it the number of kids in the family,
but from this graph you can simply see that the number of kids in a family went drastically down over the last 100 years, but the marriage age went drastically up only in the last 30 years,
ans the same behavior can be seen for the US as well.

Now for the good news.
you cannot upload your own stats to the gapminder app directly,
but Google supplies a very similar tool through its Google documents apps.
Its called motion chart and on the gapminder site you can read the short or the long explanation to how to use the tool, after testing it the outcome can be inserted into any web page and it looks something like this:

I’m still exploring this Google motion chart app,
and my main concern is that it can only be embedded in a live web page,
I still need to find a way to have this presentation as a file on my laptop, to show even if I’m not connected to the net -
If anybody knows how to do that – leave a comment here please

Technorati Tags: google, statistics

Improving Your WebSite Performance

Why is my site slow?
Why is my Bounce Rate so high?
How to improve my site’s performance?

These are some of the questions bothering web masters and site owners all the time,
here are some tools and readme’s that will:

• test your site
• find bottle necks
• measure performance from the outside
• get tips for fixing the problem areas in your site
• and knows list of issues every site is suffering from

so lets start with online freeware tools and external web site test tools.

Check if your Google Analytics is working well in your site

Who is watching the watchman?

first thing to check – are you getting the accurate statistics from your site?
is your Analytics account configured well?
are you even really using one?

SiteScan™ is a diagnostic tool from EpikOne that verifies if your Google Analytics (GA)
Tracking Code is installed properly on your website.

This scan will go over each page and verify that the Google Analytics code is embedded correctly to each of your page, this can help you diagnose if your bounce rate is high because the Analytics cant see their next click,
or maybe you should really work on your site design.

How does my code looks like from the browser?

From the firebug download page:

Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page..

This is the basic tool to verify your code as it seen from the browser side,
you can click a specific element on your page, and get a list on the right pane window of firebug,
of all the css code that is effecting this element, even if the code span multiple css files.

Why is my page so slow?

the yslow tool from yahoo will go through the elements of your page,
and by using a preset list of rules, it will tell you which parts of the website you should fix,
and what to fix.

The yslow firefox plugin needs the firefox firebug installed first.

If you want to learn more deeply on the reasons behind each of the testing rules yahoo though about you can read the very informative document from yahoo about “Best Practices for Speeding Up Your Web Site”

Feed Validator

Get your rss feed validated

AOL webpagetest

from the AOL web page test:

Pagetest allows you to provide the URL of a webpage to be tested. The test will be conducted from the location specified and you will be provided a waterfall of your page load performance as well as a comparison against an optimization checklist.

Technorati Tags: apache, google, php, wordpress