Date of Award

1-11-2021

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Information Science

First Advisor

Nitin Agarwal

Abstract

Social media plays an important role in the propagation and dissemination of ideas and thoughts. Compared to other social media platforms, blogs provide a convenient platform for users to post detailed information, engage in active discussions and share the content on other social media sites, such as Facebook and Twitter. Thus, the blogosphere has been an enormous and ever-growing part of the open-source intelligence. In order to track and monitor online social behavior particularly from blogs, the first challenging part is to mine the vast pool of unstructured data. To scale up this process and cope with the continuously changing blog structure, we propose an advanced, generic, and scalable automated blog-crawler, with ability to identify different patterns in the Hypertext Markup Language (HTML) structure of the blog pages and extract data from different blog posts. Using the crawler, we have crawled 530 blog sites with 894,856 blog posts so far.

Download

Share

COinS