What is Hacker News? Hacker News is a site started by the startup incubator Y Combinator, where user-submitted stories (known as "posts") are voted and commented upon, similar to reddit. Hacker News is extremely popular in technology and startup circles, and posts that make it to the top of Hacker News' listings can get hundreds of thousands of visitors as a result.
Dataset column description :
# importing the required libraries:
from csv import reader
opened = open('hacker_news.csv')
read = reader(opened)
hn = list(read)
# quick glance at the data:
header = hn[:1]
print(header)
print('\n')
## removing the header:
hn = hn[1:]
print(hn[:5])
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at']] [['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
We're only concerned with post titles beginning with Ask HN or Show HN, so we'll create new lists of lists containing just the data for those titles.
# to separate posts beginning with Ask HN and Show HN (and case variations) into two different lists:
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
title = row[1]
title = title.lower()
if title.startswith('show hn'):
show_posts.append(row)
elif title.startswith('ask hn'):
ask_posts.append(row)
else:
other_posts.append(row)
## to check the number of posts in each category:
template ="The number of posts in {} is {}."
print(template.format('ask_posts',len(ask_posts)) + '\n')
print(template.format('show_posts',len(show_posts)) + '\n')
print(template.format('other_posts',len(other_posts)) + '\n')
The number of posts in ask_posts is 1744. The number of posts in show_posts is 1162. The number of posts in other_posts is 17194.
# glancing at the post's data :
print(header)
print('\n')
print(ask_posts[:5])
print('\n')
print(show_posts[:5])
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at']] [['12296411', 'Ask HN: How to improve my personal website?', '', '2', '6', 'ahmedbaracat', '8/16/2016 9:55'], ['10610020', 'Ask HN: Am I the only one outraged by Twitter shutting down share counts?', '', '28', '29', 'tkfx', '11/22/2015 13:43'], ['11610310', 'Ask HN: Aby recent changes to CSS that broke mobile?', '', '1', '1', 'polskibus', '5/2/2016 10:14'], ['12210105', 'Ask HN: Looking for Employee #3 How do I do it?', '', '1', '3', 'sph130', '8/2/2016 14:20'], ['10394168', 'Ask HN: Someone offered to buy my browser extension from me. What now?', '', '28', '17', 'roykolak', '10/15/2015 16:38']] [['10627194', 'Show HN: Wio Link ESP8266 Based Web of Things Hardware Development Platform', 'https://iot.seeed.cc', '26', '22', 'kfihihc', '11/25/2015 14:03'], ['10646440', 'Show HN: Something pointless I made', 'http://dn.ht/picklecat/', '747', '102', 'dhotson', '11/29/2015 22:46'], ['11590768', 'Show HN: Shanhu.io, a programming playground powered by e8vm', 'https://shanhu.io', '1', '1', 'h8liu', '4/28/2016 18:05'], ['12178806', 'Show HN: Webscope Easy way for web developers to communicate with Clients', 'http://webscopeapp.com', '3', '3', 'fastbrick', '7/28/2016 7:11'], ['10872799', 'Show HN: GeoScreenshot Easily test Geo-IP based web pages', 'https://www.geoscreenshot.com/', '1', '9', 'kpsychwave', '1/9/2016 20:45']]
# calculating the average comments per post for ask_hn and show_hn :
## for ask_hn :
total_ask_comments = 0
for row in ask_posts:
num_com = row[4]
num_com = int(num_com)
total_ask_comments = total_ask_comments + num_com
## for show hn:
total_show_comments = 0
for row in show_posts:
num_com = row[4]
num_com = int(num_com)
total_show_comments = total_show_comments + num_com
## computing the average comments per category:
avg_ask_com = (total_ask_comments / len(ask_posts))
avg_show_com = (total_show_comments / len(show_posts))
## results :
print("The average number of comments per 'Ask HN' post is {}. ".format(round(avg_ask_com,2)) + '\n')
print("The average number of comments per 'Show HN' post is {}. ".format(round(avg_show_com,2)) + '\n')
The average number of comments per 'Ask HN' post is 14.04. The average number of comments per 'Show HN' post is 10.32.
From the above analysis we determined that on average, Ask HN posts receive morre number of comments.
As 'Ask HN' demand more user involvement by directly targeting the reader of the posts. It also shows that the readers of such posts are more involved in direct interaction.Since ask posts are more likely to receive comments, we'll focus our remaining analysis just on these posts.
i.e posts created at a certain time are more likely to attract comments.
For our analysis we need to :
## importing the required library:
import datetime as dt
# posting time of each post :
result_list = []
for row in ask_posts:
created = row[6]
num_cm = int(row[4])
result_list.append([created,num_cm])
print(result_list[:5])
[['8/16/2016 9:55', 6], ['11/22/2015 13:43', 29], ['5/2/2016 10:14', 1], ['8/2/2016 14:20', 3], ['10/15/2015 16:38', 17]]
Using two dictionaries we will calculate the average number of comments for posts created during each hour of the day.
counts_by_hour = {}
comments_by_hour = {}
for row in result_list:
dt_str = dt.datetime.strptime(row[0], "%m/%d/%Y %H:%M")
day_time = dt_str.strftime("%H")
if day_time not in counts_by_hour:
counts_by_hour[day_time] = 1
comments_by_hour[day_time] = row[1]
elif day_time in counts_by_hour:
counts_by_hour[day_time] += 1
comments_by_hour[day_time] += int(row[1])
print (counts_by_hour)
print('\n')
print (comments_by_hour)
{'09': 45, '13': 85, '10': 59, '14': 107, '16': 108, '23': 68, '12': 73, '17': 100, '15': 116, '21': 109, '20': 80, '02': 58, '18': 109, '03': 54, '05': 46, '19': 110, '01': 60, '22': 71, '08': 48, '04': 47, '00': 55, '06': 44, '07': 34, '11': 58} {'09': 251, '13': 1253, '10': 793, '14': 1416, '16': 1814, '23': 543, '12': 687, '17': 1146, '15': 4477, '21': 1745, '20': 1722, '02': 1381, '18': 1439, '03': 421, '05': 464, '19': 1188, '01': 683, '22': 479, '08': 492, '04': 337, '00': 447, '06': 397, '07': 267, '11': 641}
# To calculate the average number of comments for posts created during each hour of the day:
avg_by_hour = []
for hour in counts_by_hour and comments_by_hour:
avg_by_hour.append([hour, round((comments_by_hour[hour] / counts_by_hour[hour]),2)])
print(avg_by_hour[:5])
[['09', 5.58], ['13', 14.74], ['10', 13.44], ['14', 13.23], ['16', 16.8]]
Although we now have the results we need, this format makes it hard to identify the hours with the highest values. Let's finish by sorting the list of lists and printing the five highest values in a format that's easier to read.
swap_avg_by_hr = []
for row in avg_by_hour:
swap_avg_by_hr.append([row[1],row[0]])
print(swap_avg_by_hr[:5])
[[5.58, '09'], [14.74, '13'], [13.44, '10'], [13.23, '14'], [16.8, '16']]
## to sort the swap_avg_by_hr in descending order:
sorted_swap = sorted(swap_avg_by_hr , reverse = True)
print(sorted_swap[:5])
[[38.59, '15'], [23.81, '02'], [21.52, '20'], [16.8, '16'], [16.01, '21']]
## final values :
print( "Top 5 Hours for Ask Posts Comments : \n")
for average, hour in sorted_swap[:5]:
dt_hour = dt.datetime.strptime(str(hour), '%H')
hour = dt_hour.strftime('%H:%M')
print('{} - {} average comments per post. \n'.format(hour, average))
Top 5 Hours for Ask Posts Comments : 15:00 - 38.59 average comments per post. 02:00 - 23.81 average comments per post. 20:00 - 21.52 average comments per post. 16:00 - 16.8 average comments per post. 21:00 - 16.01 average comments per post.
In the project we analysed the dataset from Hacker Rank News posts to determine the optimal time of posting on the forum durng the day to receive the maximum number of comments from the users.i.e post at 15:00 to receive an avg of 38 comments on the post.