Free data journalism tools - time lines

This post starts series of posts with free tools for data journalism. To make everything clear, I have nothing against paid apps but I just wanted to show that having strong willingness to work with data and no budget is enough to build a working place for a data journalist / yourself / data blogger / student / random person who likes data. In more romantic words, love for data is everything, you don't need more :)

Creating and saving path as KML file in Google Earth

You can use KML files to store routes of your trips, bird migration, geographical discoveries or whatever you want. Later you can use KML files to present created geo data in Google Fusion Tables.

How to create KML file to store your geo data

You can use KML files to create maps in Google Fusion Tables. 

Comments to the tutorial: I keep everything in folders, it's not a must but when you have a few polygons or lines or points it's easier to keep order. I segregate everything by type, so for example polygons and points are never in the same folder, I do this just in case if I wanted to export KML to shape file.

This tutorial is useful when you want to create geographic files of places that are hard to find in ready geo files. It's recommended for smaller projects, as everything is done by hand. Easy to imagine that mapping all buildings in a selected country isn't this case :)

Shape file directory

It's a list from my perspective, I work mainly with geo data for Europe and Asian post Soviet Union countries. I wanted to keep all my sources in one place so I started this list, which I am planning to update when I find something interesting. I focus on open source shape files.

If you came across a website with shape files and you want to share it with me, it will be the happiest day of my life.

Велодорожки Москвы / Cycle lanes of Moscow

Портал открытых данных правительства Москвы опубликовал географические координаты московских велодорожек.
Общая длина велодорожек состоит 111 км. Самая длинная велодорожка находится в парке «Лосиный Остров».

Я хотела сделать карту поинтереснее и добавить другие данные к велосипедным геоданным, но не нашла ничего подходящего.

На моей карте можно включить Street View чтобы увидеть фотографии.
========================================================

English version :)

Reshaping data in Data Wrangler

Data Wrangler is a hero of this tutorial. It's free online tool to transform data.

There are three reasons why it's worth to use Data Wrangler:

• it's very easy to use
• although it's simple it offers many useful functions
• the app's done by nice people

The app has limitations, Wrangler only supports up to 40 columns and 1000 rows. If you are planning to work with something bigger than that, then you have to divide your data. This limitation is the most painful part of this tutorial so you can relax and keep reading, now there will be only positive things.

Something old, something new

I'm totally addicted to archaeological documentaries. My favourite archaeological site is Göbekli Tepe, google it, it's just WOW. In July 2014 Klaus Schmidt, who headed the excavations at the site, passed away. I started to thinking about doing something in this field. I wanted to analyse some data devoted to Göbekli Tepe, but I couldn't find anything. Archaeological data published in the internet are rarity.

Simplifying geometries of polygon shape files in QGIS's SimpliPy

First you have to install SimpliPy

Note that the plug-in says: You must have installed Shapely 1.2.18 or you will get an error installing or using this plug-in. Shapely is open source.

==============================================
Here you can learn what simplifying geometries of shape files is.
==============================================

How the world has changed from 1960, population comparator

I was playing with population data but I wanted to do something different so I took into account data for a long period.

I prepared population comparator, a tool to compare how the growth of population proceed in countries. Don't get wild and don't select too many countries as it will be impossible to see anything.

Cycling in London

I've done a cycling map of London.
Not everything was as I planned. I was searching for a line shape file of the London's cycle lanes, as logical would be adding length of cycle lanes per ward or any data set saying about an average commuting distance to work. I asked Transport for London and a few cycling organizations based in London for the shape file but nobody was able to help me.

Simplifying geometries of polygon shape files

If your map works very slowly and you have to wait for a respond for a loooooooooong time, it's a clear sign that you need to simplify the geometries of the shape files.

How to export a point shape file to csv in QGIS

A point shape file in QGIS looks like this:

Maps in Tableau 8.2 - street level

There're new maps in Tableau 8.2. Now you can choose between Tableau (the new maps) and Tableau Classic (the old ones, not medieval maps but just from 8.1 version).

Tableau Mapping Template for London Boroughs

Feel free to use this template in your Tableau projects, just remember about placing the information about the owner of the original shape file.

You can check Tableau template for London wards too.

Tableau Mapping Template for London Wards

If you've searched for a Tableau mapping template for London, you can stop googling :) If you need other templates for the UK, you should check http://tableaumapping.bi

Tableau Mapping is a website where Tableau users share their maps created from open shape files, feel free to join them. Situation with shape files is complex as only in some countries the Statistics Offices publish them for free, so if you spot such file it's worth to give the link to Tableau Mapping.

Devil's tooltip (ubertip) in Tableau

The official name of this tooltip's the ubertip, but the devil's tooltip shows its real nature.

Ubertips appear after clicking on row or column headers, when clicking on a mark or data item in the view, they don't appear.

Simplifying shapefiles in MapShaper

Maps should be simple, simple to make and simple to use.

I simplify shape files to use them in Tableau but you can use them in other apps too.

Why should you do this?

Speaking in simple words Tableau draws maps by joining nodes with lines in order given by ShapeToTab. If there are many nodes Tableau has to work harder and performs slower. Besides that it is a great way of spending free time :)

==============================================
Here you can learn what simplifying geometries of shape files is.
==============================================

Arithmeum, part 1

I am planning to add a few posts about data in the real world, yes life and data also exist outside the internet. I have no idea how I will achieve this as I do not go out so often. Keep your fingers crossed :) 

If you are into a data analysis, you have argued with your data more than once. Pieces of data software have made you crazy. Be honest, you have insulted Excel or any other application. After reading this magical blog post you will appreciate modern data programs. You will purr like a kitty thinking of calculating on your computer.

I have visited Arithmeum, a museum devoted to calculating methods and machines. It is placed in Bonn, Germany. 

Население наиболее многочисленных национальностей по субъектам Российской Федерации

Если тебе интересна эта тема тоже стоит посмотреть эту визуализацию данных.

Я сравнила данные с переписи населения 2010 года с 1926 годом.

English version is here. 

Poor diet

I've decided to spend one week living on the edge of poverty to get the data for the Tableau's Quantified Self Viz Contest. My food budget was $2.35 per day (it's calculated by government organization), I stayed in Poland so prices are from there.

Ethnic minorities in Russia

I've found the old census data from 1926 and decided to give them a new life by comparing to the newest census.

I haven't included all ethnic minorities in Russia as there're some mistakes in the old censuses, I've checked all censuses between 1926 and 2010, some ethnic groups're in a few first censuses then they just disappeared and appeared again. Of course it made me crazy but life is like this. As a result of this predictable problem with the old censuses, I decide to use in my visualization only the biggest ethnic groups.

Pyramid in Tableau

In the ancient Egypt building pyramids was a team work, now you can create a pyramid on your own. Though you can't use it as a grave, what can be a disadvantage...

Russian text in Excel lost when saved as a CSV file

This problem could happen to everyone, even to you, maybe not to everyone as not everyone works with Russian text in Excel lol

The problem appears only when you're trying to save the text in a language other than your system's language, so if your system is in Russian and the file consists Russian text, you don't need to worry, this tutorial is not for you.

Ethnic groups in Russia

Russia is home to over 185 ethnic groups, population of these groups vary from millions to few thousands. In some regions Russians are in the minority.

This topic is rich in data so later I will post more works.

How to create polygon map based on image in Tableau - plan B

This tutorial is devoted to creating polygon map in Tableau based on image, I've done the tutorial giving similar results here.
Maybe you will find this way easier.

It's a good way of mapping places that aren't strongly connected to Longitude/Latitude. Shops in shopping centres have their geographical coordinates but of course nobody uses them, so map consisting general shapes without details is fine in such situation.

Top 30 countries with the highest military expenditure in 2013

Stockholm International Peace Research Institute (SIPRI) has published military expenditure for 2013. Global military expenditure totalled $1.75 trillion.

Just two states, USA and China, made nearly half of all military spending.

How to create polygon map based on image in Tableau

This tutorial is based on the same way of thinking which is used by ShapeToTab. Creating ShapeToTab results without using ShapeToTab lol. In other words it is very easy to do :) 

We will be creating a map of shopping centre.

Gazprom's natural gas export structure to European and FSU countries in 2012

The largest buyers of the Russian gas were Germany (16% of Gazprom's export), Ukraine (15%), Turkey (12%) and Belarus (9%).

Gazprom Group in 2012 sold in total 151 bcm of natural gas to European countries and 66 bcm to FSU countries.

The data have been published by Gazprom in Gazprom in Figures 2008-2012. 

Die grössten Waffenexporteure und Importeure 2009-2013

The easiest and cheapest way to learn Tableau

My tips for people who want to learn Tableau without any budget :)

1. Training & Tutorials on Tableau's website

Select 'On Demand', you have to create an account but it's free and you don't need to be a Tableau customer. The tutorials are selected into few groups Intro, Advanced, Chart Types, Server. It's the best place to start learning.

Check 'Live Online' out too.

Tableau's Knowledge Base – hundreds of tutorials are divided by topic.

Each short tutorial gives answer to a problem, it's good to check the titles and do tutorials which can be useful in your work. The list look like a never ending story but no reason to be in stress as most of things you will learn doing 'On Demand' tutorials.

How to get Latitude and Longitude (geographic coordinates)

You need Latitude and Longitude of a place (address, city, etc.).

There is a good tool for each life situation, so let's check the possibilities.

1. You have a few thousands of addresses and you need its geographic coordinates. The answer is Batch Geocoding.

Total % of internet users

Cross-stitching is good for everything also for presenting data. One of the most sophisticated ways of working with data ;)

In 1994 five people out of 1000 had access to the internet, in 2012 36 out of 100.

We are in the point when one third of world's population uses internet, it is symbolised by the person in the pink t-shirt (one out of three).

Bill Gates, poverty and snack

Bill Gates wrote to me to inform how the world has changed since '89. To be honest he just tweeted :) so no reason to keep this confidential: Gates annual letter.

The most interesting to me was the chart showing change in percentage of people living on less than $1.25 per day, so I decided to create my version, check both.

QGIS basic tutorial 2 - merging administrative divisions

Let's analyse a situation, you have a data set and you have something, what looks like the matching shape file, you think cheerfully that the data set will fit to the shape file like a hand to the glove. Unfortunately, life isn't so easy. Some shape files aren't updated regularly, it is common that administrative districts with low population density are joined to the neighbouring districts. It is worth to compare the number of districts in your data sets with the shape file. If you see any gap, try to find information about changes, so you will know which districts should be merged.

QGIS basic tutorial 1 - attribute table, sorting, filtering

I use QGIS to prepare shape files for creation of maps in Tableau. Some shape files demand changes to match my Tableau projects so I decided to share few tips how to work with them.

In the tutorial I work with QGIS 2.0.1. and the shape file of Ukrainian administrative districts from GADM.

Terrorist attacks in Russia since 1991

The researchers at the University of Maryland have created the world's largest database of terrorists attacks.
Between 1991 and 2012, there were 1,895 attacks in Russia.

In my dataviz I used data published by the Guardian.
I would like to add the data for 2013, hope to get them somewhere...