Free data journalism tools - data extracting and cleaning

Extracting data from PDF

There are plenty of people who strongly believe that the best place for their data is PDF, everyone does it, statistical offices, international organizations. There is only one reason behind it, their pleasure, yes they find it pleasurable to imagine people who try to extract their data from PDF.

Creating and saving path as KML file in Google Earth

You can use KML files to store routes of your trips, bird migration, geographical discoveries or whatever you want. Later you can use KML files to present created geo data in Google Fusion Tables.

How to create KML file to store your geo data

You can use KML files to create maps in Google Fusion Tables. 

Comments to the tutorial: I keep everything in folders, it's not a must but when you have a few polygons or lines or points it's easier to keep order. I segregate everything by type, so for example polygons and points are never in the same folder, I do this just in case if I wanted to export KML to shape file.

This tutorial is useful when you want to create geographic files of places that are hard to find in ready geo files. It's recommended for smaller projects, as everything is done by hand. Easy to imagine that mapping all buildings in a selected country isn't this case :)

Shape file directory

It's a list from my perspective, I work mainly with geo data for Europe and Asian post Soviet Union countries. I wanted to keep all my sources in one place so I started this list, which I am planning to update when I find something interesting. I focus on open source shape files.

If you came across a website with shape files and you want to share it with me, it will be the happiest day of my life.

Reshaping data in Data Wrangler

Data Wrangler is a hero of this tutorial. It's free online tool to transform data.

There are three reasons why it's worth to use Data Wrangler:

• it's very easy to use
• although it's simple it offers many useful functions
• the app's done by nice people

The app has limitations, Wrangler only supports up to 40 columns and 1000 rows. If you are planning to work with something bigger than that, then you have to divide your data. This limitation is the most painful part of this tutorial so you can relax and keep reading, now there will be only positive things.

Simplifying geometries of polygon shape files in QGIS's SimpliPy

First you have to install SimpliPy

Note that the plug-in says: You must have installed Shapely 1.2.18 or you will get an error installing or using this plug-in. Shapely is open source.

==============================================
Here you can learn what simplifying geometries of shape files is.
==============================================

Simplifying geometries of polygon shape files

If your map works very slowly and you have to wait for a respond for a loooooooooong time, it's a clear sign that you need to simplify the geometries of the shape files.

How to export a point shape file to csv in QGIS

A point shape file in QGIS looks like this:

Maps in Tableau 8.2 - street level

There're new maps in Tableau 8.2. Now you can choose between Tableau (the new maps) and Tableau Classic (the old ones, not medieval maps but just from 8.1 version).

Devil's tooltip (ubertip) in Tableau

The official name of this tooltip's the ubertip, but the devil's tooltip shows its real nature.

Ubertips appear after clicking on row or column headers, when clicking on a mark or data item in the view, they don't appear.

Simplifying shapefiles in MapShaper

Maps should be simple, simple to make and simple to use.

I simplify shape files to use them in Tableau but you can use them in other apps too.

Why should you do this?

Speaking in simple words Tableau draws maps by joining nodes with lines in order given by ShapeToTab. If there are many nodes Tableau has to work harder and performs slower. Besides that it is a great way of spending free time :)

==============================================
Here you can learn what simplifying geometries of shape files is.
==============================================

Pyramid in Tableau

In the ancient Egypt building pyramids was a team work, now you can create a pyramid on your own. Though you can't use it as a grave, what can be a disadvantage...

Russian text in Excel lost when saved as a CSV file

This problem could happen to everyone, even to you, maybe not to everyone as not everyone works with Russian text in Excel lol

The problem appears only when you're trying to save the text in a language other than your system's language, so if your system is in Russian and the file consists Russian text, you don't need to worry, this tutorial is not for you.

How to create polygon map based on image in Tableau - plan B

This tutorial is devoted to creating polygon map in Tableau based on image, I've done the tutorial giving similar results here.
Maybe you will find this way easier.

It's a good way of mapping places that aren't strongly connected to Longitude/Latitude. Shops in shopping centres have their geographical coordinates but of course nobody uses them, so map consisting general shapes without details is fine in such situation.

How to create polygon map based on image in Tableau

This tutorial is based on the same way of thinking which is used by ShapeToTab. Creating ShapeToTab results without using ShapeToTab lol. In other words it is very easy to do :) 

We will be creating a map of shopping centre.

The easiest and cheapest way to learn Tableau

My tips for people who want to learn Tableau without any budget :)

1. Training & Tutorials on Tableau's website

Select 'On Demand', you have to create an account but it's free and you don't need to be a Tableau customer. The tutorials are selected into few groups Intro, Advanced, Chart Types, Server. It's the best place to start learning.

Check 'Live Online' out too.

Tableau's Knowledge Base – hundreds of tutorials are divided by topic.

Each short tutorial gives answer to a problem, it's good to check the titles and do tutorials which can be useful in your work. The list look like a never ending story but no reason to be in stress as most of things you will learn doing 'On Demand' tutorials.

How to get Latitude and Longitude (geographic coordinates)

You need Latitude and Longitude of a place (address, city, etc.).

There is a good tool for each life situation, so let's check the possibilities.

1. You have a few thousands of addresses and you need its geographic coordinates. The answer is Batch Geocoding.

QGIS basic tutorial 2 - merging administrative divisions

Let's analyse a situation, you have a data set and you have something, what looks like the matching shape file, you think cheerfully that the data set will fit to the shape file like a hand to the glove. Unfortunately, life isn't so easy. Some shape files aren't updated regularly, it is common that administrative districts with low population density are joined to the neighbouring districts. It is worth to compare the number of districts in your data sets with the shape file. If you see any gap, try to find information about changes, so you will know which districts should be merged.

QGIS basic tutorial 1 - attribute table, sorting, filtering

I use QGIS to prepare shape files for creation of maps in Tableau. Some shape files demand changes to match my Tableau projects so I decided to share few tips how to work with them.

In the tutorial I work with QGIS 2.0.1. and the shape file of Ukrainian administrative districts from GADM.

How to create map in Tableau with ShapeToTab

Prepare environment for work in ShapeToTab (created by Richard Leeke)

download ShapeToTab from http://dl.dropbox.com/u/59458890/ShapeToTab.zip 

download release-1600-gdal-1-9-0-mapserver-6-0-1 from http://www.gisinternals.com/sdk/PackageList.aspx?file=release-1600-gdal-1-9-0-mapserver-6-0-1.zip
Unzip both and place them on your computer.

ShapeToTab consists of (my system is not in English, sorry):