In this tutorial we look at how data can be grouped (for example by year) and how the resulting grouped data can be viewed and analysed.

We begin by reading data from IMDB into the movies_df dataframe. We then show the first 5 rows in this dataframe.

Next, for convenience, we rename all of the column titles so that they are all lowercase.

Grouping by year

The data is grouped by year. This creates a grouped dataframe – this is essentially a collection of dataframes that are index by the year.

Inspecting a particular year

If wanted to view the information for a particular group (in this case for a particular year) we can use the get_group function. The code below will get the dataframe for 2009 and show the first 5 entries.

Sorting values

In the following code we use the sort_values function to sort the movies released in 2009 into from best to worst (using ascending=False) and from worst to best (using ascending=True or omitting this option).

The first three entries from each of these sorted dataframes are displayed.

Summary information

Summary calculations can be done on the grouped data. For example we can calculate the mean rating for each year using the mean function.

Similarly, the maximum metascore value for each year can be calculated using the max function. Whilst this tables shows us what the top metascore was for each year, it does not show the associated movie title (or any other associated information). Other aggregate functions also have a similar limitation. We will show in the next section how to create a more detailed summary of the best (or worst) movies.

Top rated movie for each year

In the following code we create a new dataframe showing the top rated movie from each year from 2006 up until 2016. For each entry we show the year of release, movie title, rating and (for comparison) the metascore.

To create this dataframe we begin by creating a new dictionary and add entries for the four columns of interest noted above. Then for each group in the grouped data frame we get the row with the highest rating. This is done by sorting the group on the rating in descending order, then getting the first row in the sorted group.

Best movies by metascore

For comparison we also create a dataframe to store the top movie (ranked by metascore) for each year.

Summarising the distributions of ratings using a boxplot

In the following code we show how boxplots can be created for each year of release and shown on the same graph for easy comparison of movie ratings across each year.

Grouping by genre

Next we group the movies by genre. However we note that in the original datafile many of the movies had multiple comma separated genres. The following code creates a new data frame with movies that had multiple genres split across multiple rows with one genre list per row.

Grouping by actor

In the following code a new dataframe is created containing one actor per row. Before the individual actor names are added to the dataframe any leading spaces at the beginning of the line are removed using the lstrip function.

Sorting by number of movie appearances

The count method is used to determine how many movies each actor has appeared in. The list is sorted from highest to lowest.

Ranking actors

Next we rank actors based on the average metascore of all movies they have appeared in.

In this case the list is skewed by actors who have only appeared in a single movie.

Adding a minimum number of movies filter

To ensure that the ranking of actors is not skewed by those who have only appeared in a small number of movies, we filter the data to only include actors who have appeared in more than 6 movies.

The high flyers

The stinkers