This data set is Hacker News posts from the last 12 months (up to September 26 2016).
It includes the following columns:
title
: title of the post (self explanatory)
url
: the url of the item being linked to
num_points
: the number of upvotes the post received
num_comments
: the number of comments the post received
author
: the name of the account that made the post
created_at
: the date and time the post was made (the time zone is Eastern Time in the US)
In this project we are specifically interested in posts whose titles begin with either Ask HN
or Show HN
. Users submit Ask HN posts to ask the Hacker News community a specific question. Likewise, users submit Show HN
posts to show the Hacker News community a project, product, or just generally something interesting.
We'll compare these two types of posts to determine the following:
Ask HN
or Show HN
receive more comments on average?Read the file in as a list of lists and extract headers
from csv import reader
opened_file = open("hacker_news.csv")
read_file = reader(opened_file)
hn = list(read_file)
headers = hn[0]
hn= hn[1:]
print(headers)
print(hn[:5])
['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'] [['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
ask_posts =[]
show_posts =[]
other_posts =[]
for i in hn:
title = i[1]
if title.lower().startswith('ask hn'):
ask_posts.append(i)
elif title.lower().startswith('show hn'):
show_posts.append(i)
else:
other_posts.append(i)
ask_count = len(ask_posts)
show_count = len(show_posts)
others_count = len(other_posts)
print(" No. of ask posts:",ask_count,'\n',"No. of show posts:", show_count,'\n',"No. of other posts:", others_count)
No. of ask posts: 1744 No. of show posts: 1162 No. of other posts: 17194
## ask-posts
total_ask_comments = 0
for posts in ask_posts:
num_comments = int(posts[4]) # convert to int to ease the calculation
total_ask_comments += num_comments
avg_ask_comments = total_ask_comments/ask_count
print("Average comments in ask posts:",avg_ask_comments)
## show-posts
total_show_comments = 0
for posts in show_posts:
num_comments = int(posts[4]) # convert to int to ease the calculation
total_show_comments += num_comments
avg_show_comments = total_show_comments/show_count
print("Average comments in show posts:",avg_show_comments)
TypeErrorTraceback (most recent call last) <ipython-input-35-abb01e75d481> in <module>() 1 ## ask-posts 2 total_ask_comments = 0 ----> 3 for posts in ask_posts.sort(): 4 num_comments = int(posts[4]) # convert to int to ease the calculation 5 total_ask_comments += num_comments TypeError: 'NoneType' object is not iterable
From the average values as seen above, we can conclude that on an average the ask posts received approx. 14 comments more than the show posts in the HN community.
Next, we'll determine if ask posts created at a certain time are more likely to attract comments. We'll use the following steps to perform this analysis:
Calculate the amount of ask posts created in each hour of the day, along with the number of comments received.
Calculate the average number of comments ask posts receive by hour created.
# Calculate the amount of ask posts created during
# each hour of day and the number of comments received.
import datetime as dt
result_list = []
counts_by_hour={}
comments_by_hour ={}
date_format = "%m/%d/%Y %H:%M"
for posts in ask_posts:
result_list.append(
[posts[6], int(posts[4])]
)
for posts in result_list:
created_at = posts[0]
comments = int(posts[1])
hour = dt.datetime.strptime(created_at, date_format).strftime("%H")
if hour in counts_by_hour:
counts_by_hour[hour] += 1
comments_by_hour[hour] += comments
else:
counts_by_hour[hour] = 1
comments_by_hour[hour] = comments
print("Comments by hour \n",comments_by_hour)
print("\nCount by hour \n",counts_by_hour)
Comments by hour {'15': 4477, '16': 1814, '09': 251, '05': 464, '21': 1745, '22': 479, '20': 1722, '17': 1146, '12': 687, '02': 1381, '10': 793, '00': 447, '06': 397, '01': 683, '14': 1416, '08': 492, '03': 421, '07': 267, '23': 543, '13': 1253, '18': 1439, '19': 1188, '04': 337, '11': 641} Count by hour {'15': 116, '16': 108, '09': 45, '05': 46, '21': 109, '22': 71, '20': 80, '17': 100, '12': 73, '02': 58, '10': 59, '00': 55, '06': 44, '01': 60, '14': 107, '08': 48, '03': 54, '07': 34, '23': 68, '13': 85, '18': 109, '19': 110, '04': 47, '11': 58}
Ask HN
posts during each hour of the day¶# Calculate the average amount of comments
# ask posts created at each hour of the day.
avg_by_hour=[]
for hour in comments_by_hour:
avg_by_hour.append([hour,comments_by_hour[hour]/counts_by_hour[hour]])
# print(avg_by_hour)
avg_by_hour
[['15', 38.5948275862069], ['16', 16.796296296296298], ['09', 5.5777777777777775], ['05', 10.08695652173913], ['21', 16.009174311926607], ['22', 6.746478873239437], ['20', 21.525], ['17', 11.46], ['12', 9.41095890410959], ['02', 23.810344827586206], ['10', 13.440677966101696], ['00', 8.127272727272727], ['06', 9.022727272727273], ['01', 11.383333333333333], ['14', 13.233644859813085], ['08', 10.25], ['03', 7.796296296296297], ['07', 7.852941176470588], ['23', 7.985294117647059], ['13', 14.741176470588234], ['18', 13.20183486238532], ['19', 10.8], ['04', 7.170212765957447], ['11', 11.051724137931034]]
swap_avg_by_hour = []
for i in avg_by_hour:
swap_avg_by_hour.append([i[1],i[0]])
#print(swap_avg_by_hour)
sorted_swap = sorted(swap_avg_by_hour, reverse = True)
print("Top 5 Hours for `Ask` Posts Comments \n")
for hr in sorted_swap:
avg = hr[0]
time = hr[1]
time = dt.datetime.strftime(time, "%H:%M")
print(("{0}:{avg:.2f} average comments per post").format(time,avg))
TypeErrorTraceback (most recent call last) <ipython-input-71-e3928d56c0d8> in <module>() 5 #print(swap_avg_by_hour) 6 ----> 7 sorted_swap = sorted(swap_avg_by_hour, reverse = True) 8 9 print("Top 5 Hours for `Ask` Posts Comments \n") TypeError: sorted() got an unexpected keyword argument 'reverse'