Hacker News is a site started by the startup incubator Y Combinator, where user-submitted stories (known as "posts") are voted and commented upon, similar to reddit. Hacker News is extremely popular in technology and startup circles, and posts that make it to the top of Hacker News' listings can get hundreds of thousands of visitors as a result.
You can find the data set here, but note that it has been reduced from almost 300,000 rows to approximately 20,000 rows by removing all submissions that did not receive any comments, and then randomly sampling from the remaining submissions. Below are descriptions of the columns:
We're specifically interested in posts whose titles begin with eitherAsk HN
or Show HN
. Users submit Ask HN
posts to ask the Hacker News community a specific question.
Likewise, users submit Show HN
posts to show the Hacker News community a project, product, or just generally something interesting.
We'll compare these two types of posts to determine the following:
Ask HN
or Show HN
receive more comments on average?# Read in the data.
import csv
file = open('hacker_news.csv')
hn = list(csv.reader(file))
print(hn[:5])
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'], ['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01']]
headers = hn[0]
hn = hn[1:]
print(headers)
print(hn[:5])
['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'] [['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
Now that we've removed the headers from hn, we're ready to filter our data. Since we're only concerned with post titles beginning with Ask HN
or Show HN
, we'll create new lists of lists containing just the data for those titles.
ask_posts = []
show_posts =[]
other_posts = []
for post in hn:
title = post[1]
if title.lower().startswith("ask hn"):
ask_posts.append(post)
elif title.lower().startswith("show hn"):
show_posts.append(post)
else:
other_posts.append(post)
print(len(ask_posts))
print(len(show_posts))
print(len(other_posts))
1744 1162 17194
Let's determine if ask posts or show posts receive more comments on average
total_ask_comments = 0
for post in ask_posts:
total_ask_comments += int(post[4])
avg_ask_comments = total_ask_comments / len(ask_posts)
print(avg_ask_comments)
total_show_comments = 0
for post in show_posts:
total_show_comments += int(post[4])
avg_show_comments = total_show_comments / len(show_posts)
print(avg_show_comments)
14.038417431192661 10.31669535283993
Ask posts in our sample receive around 14 comments on average, while show posts receive around 10.
Since ask posts are more likely to receive comments, we'll focus our remaining analysis just on these posts.
Next, we'll determine if ask posts created at a certain time are more likely to attract comments. We'll use the following steps to perform this analysis:
import datetime as dt
result_list = []
for post in ask_posts:
result_list.append(
[post[6], int(post[4])]
)
counts_by_hour = {}
comments_by_hour = {}
for row in result_list:
date = row[0]
comment = row[1]
d_format = "%m/%d/%Y %H:%M"
time = dt.datetime.strptime(date, d_format).strftime("%H")
if time not in counts_by_hour:
counts_by_hour[time] = 1
comments_by_hour[time] = comment
else:
counts_by_hour[time] += 1
comments_by_hour[time] += comment
print(comments_by_hour)
{'09': 251, '13': 1253, '10': 793, '14': 1416, '16': 1814, '23': 543, '12': 687, '17': 1146, '15': 4477, '21': 1745, '20': 1722, '02': 1381, '18': 1439, '03': 421, '05': 464, '19': 1188, '01': 683, '22': 479, '08': 492, '04': 337, '00': 447, '06': 397, '07': 267, '11': 641}
avg_by_hour = []
for hour in comments_by_hour:
avg_by_hour.append([hour, comments_by_hour[hour] / counts_by_hour[hour]])
print(avg_by_hour)
[['09', 5.5777777777777775], ['13', 14.741176470588234], ['10', 13.440677966101696], ['14', 13.233644859813085], ['16', 16.796296296296298], ['23', 7.985294117647059], ['12', 9.41095890410959], ['17', 11.46], ['15', 38.5948275862069], ['21', 16.009174311926607], ['20', 21.525], ['02', 23.810344827586206], ['18', 13.20183486238532], ['03', 7.796296296296297], ['05', 10.08695652173913], ['19', 10.8], ['01', 11.383333333333333], ['22', 6.746478873239437], ['08', 10.25], ['04', 7.170212765957447], ['00', 8.127272727272727], ['06', 9.022727272727273], ['07', 7.852941176470588], ['11', 11.051724137931034]]
swap_avg_by_hour = []
for row in avg_by_hour:
swap_avg_by_hour.append([row[1], row[0]])
print(swap_avg_by_hour)
print("\n")
sorted_swap = sorted(swap_avg_by_hour, reverse = True)
print(sorted_swap)
print("\n")
print("Top 5 Hours for Ask Posts Comments")
for avg, hour in sorted_swap[:5]:
time = dt.datetime.strptime(hour, "%H").strftime("%H:%M")
print( "{}: {:.2f} average comments per post".format(time, avg))
[[5.5777777777777775, '09'], [14.741176470588234, '13'], [13.440677966101696, '10'], [13.233644859813085, '14'], [16.796296296296298, '16'], [7.985294117647059, '23'], [9.41095890410959, '12'], [11.46, '17'], [38.5948275862069, '15'], [16.009174311926607, '21'], [21.525, '20'], [23.810344827586206, '02'], [13.20183486238532, '18'], [7.796296296296297, '03'], [10.08695652173913, '05'], [10.8, '19'], [11.383333333333333, '01'], [6.746478873239437, '22'], [10.25, '08'], [7.170212765957447, '04'], [8.127272727272727, '00'], [9.022727272727273, '06'], [7.852941176470588, '07'], [11.051724137931034, '11']] [[38.5948275862069, '15'], [23.810344827586206, '02'], [21.525, '20'], [16.796296296296298, '16'], [16.009174311926607, '21'], [14.741176470588234, '13'], [13.440677966101696, '10'], [13.233644859813085, '14'], [13.20183486238532, '18'], [11.46, '17'], [11.383333333333333, '01'], [11.051724137931034, '11'], [10.8, '19'], [10.25, '08'], [10.08695652173913, '05'], [9.41095890410959, '12'], [9.022727272727273, '06'], [8.127272727272727, '00'], [7.985294117647059, '23'], [7.852941176470588, '07'], [7.796296296296297, '03'], [7.170212765957447, '04'], [6.746478873239437, '22'], [5.5777777777777775, '09']] Top 5 Hours for Ask Posts Comments 15:00: 38.59 average comments per post 02:00: 23.81 average comments per post 20:00: 21.52 average comments per post 16:00: 16.80 average comments per post 21:00: 16.01 average comments per post
According to the documentation of the dataset, the timezone used is US Eastern Time. One should post around 3 pm est to have the greatest chance of receiving comments.
total_ask_points = 0
for post in ask_posts:
total_ask_points += int(post[3])
avg_ask_points = total_ask_points / len(ask_posts)
print(avg_ask_points)
total_show_points = 0
for post in show_posts:
total_show_points += int(post[3])
avg_show_points = total_show_points / len(show_posts)
print(avg_show_points)
15.061926605504587 27.555077452667813
Ask posts in our sample receive around 15 points on average, while show posts receive around 27.5.
result_list_points = []
for post in show_posts:
result_list_points.append(
[post[6], int(post[3])]
)
counts_by_hour_2 = {}
points_by_hour = {}
for row in result_list_points:
date = row[0]
points = row[1]
d_format = "%m/%d/%Y %H:%M"
time = dt.datetime.strptime(date, d_format).strftime("%H")
if time not in counts_by_hour_2:
counts_by_hour_2[time] = 1
points_by_hour[time] = points
else:
counts_by_hour_2[time] += 1
points_by_hour[time] += points
print(points_by_hour)
{'14': 2187, '22': 1856, '18': 2215, '07': 494, '20': 1819, '05': 104, '16': 2634, '19': 1702, '15': 2228, '03': 679, '17': 2521, '06': 375, '02': 340, '13': 2438, '08': 519, '21': 866, '04': 386, '11': 1480, '12': 2543, '23': 1526, '09': 553, '01': 700, '10': 681, '00': 1173}
avg_points_by_hour = []
for hour in points_by_hour:
avg_points_by_hour.append([hour, points_by_hour[hour] / counts_by_hour_2[hour]])
print(avg_points_by_hour)
[['14', 25.430232558139537], ['22', 40.34782608695652], ['18', 36.31147540983606], ['07', 19.0], ['20', 30.316666666666666], ['05', 5.473684210526316], ['16', 28.322580645161292], ['19', 30.945454545454545], ['15', 28.564102564102566], ['03', 25.14814814814815], ['17', 27.107526881720432], ['06', 23.4375], ['02', 11.333333333333334], ['13', 24.626262626262626], ['08', 15.264705882352942], ['21', 18.425531914893618], ['04', 14.846153846153847], ['11', 33.63636363636363], ['12', 41.68852459016394], ['23', 42.388888888888886], ['09', 18.433333333333334], ['01', 25.0], ['10', 18.916666666666668], ['00', 37.83870967741935]]
swap_avg_points_by_hour = []
for row in avg_points_by_hour:
swap_avg_points_by_hour.append([row[1], row[0]])
print(swap_avg_points_by_hour)
print("\n")
sorted_swap_points = sorted(swap_avg_points_by_hour, reverse = True)
print(sorted_swap_points)
print("\n")
print("Top 5 Hours for Show Posts Points")
for avg, hour in sorted_swap_points[:5]:
time = dt.datetime.strptime(hour, "%H").strftime("%H:%M")
print( "{}: {:.2f} average points per post".format(time, avg))
[[25.430232558139537, '14'], [40.34782608695652, '22'], [36.31147540983606, '18'], [19.0, '07'], [30.316666666666666, '20'], [5.473684210526316, '05'], [28.322580645161292, '16'], [30.945454545454545, '19'], [28.564102564102566, '15'], [25.14814814814815, '03'], [27.107526881720432, '17'], [23.4375, '06'], [11.333333333333334, '02'], [24.626262626262626, '13'], [15.264705882352942, '08'], [18.425531914893618, '21'], [14.846153846153847, '04'], [33.63636363636363, '11'], [41.68852459016394, '12'], [42.388888888888886, '23'], [18.433333333333334, '09'], [25.0, '01'], [18.916666666666668, '10'], [37.83870967741935, '00']] [[42.388888888888886, '23'], [41.68852459016394, '12'], [40.34782608695652, '22'], [37.83870967741935, '00'], [36.31147540983606, '18'], [33.63636363636363, '11'], [30.945454545454545, '19'], [30.316666666666666, '20'], [28.564102564102566, '15'], [28.322580645161292, '16'], [27.107526881720432, '17'], [25.430232558139537, '14'], [25.14814814814815, '03'], [25.0, '01'], [24.626262626262626, '13'], [23.4375, '06'], [19.0, '07'], [18.916666666666668, '10'], [18.433333333333334, '09'], [18.425531914893618, '21'], [15.264705882352942, '08'], [14.846153846153847, '04'], [11.333333333333334, '02'], [5.473684210526316, '05']] Top 5 Hours for Show Posts Points 23:00: 42.39 average points per post 12:00: 41.69 average points per post 22:00: 40.35 average points per post 00:00: 37.84 average points per post 18:00: 36.31 average points per post
For show posts one should post around 11 pm est to have the greatest chance of receiving the most points.