Analyzing Headlines for Amusement and Minimal Financial Gain
### From a Laptop to a Raspberry Pi: The Evolution of a News Analysis System
In the early 2000s, a pioneering individual embarked on a unique project: creating a news corpus analysis system. This system, designed to compare the occurrence of words over time, track news cycles, and identify trends, would evolve into a powerful tool for media analysis.
The journey began with the author using PHP5 and MySQL to build a website for their project in 2005. However, as the corpus expanded, problems arose with the MySQL database. In response, the author found an alternative by utilising the most basic operating system feature, the filesystem, to store the data.
As the decade progressed, cloud services such as AWS emerged, offering the opportunity to rent cloud buckets for minimal cost. This development allowed the system to be transitioned from a hard-working x86 laptop to a low-powered Raspberry Pi with a USB hard disk.
The system was designed to predict big stories based on sudden drops in coverage of a topic, and to identify which outlets covered a story more than others, aiding in the determination of lobby influence. It also aimed to discern the most commonly used language, referred to as the "cider," from less common language, known as the "cyder."
The author's text analysis experiment evolved into something more, providing insights into current events and teaching the author about various technical aspects, including statistics, language, text parsing, and hard drive management. This journey not only resulted in a powerful media analysis tool but also fostered a deeper understanding of the digital world.
In the 1990s, web hosting was primarily measured by storage space, with little consideration given to processing power. However, by the 2000s, the price of storage had significantly dropped while the cost of processing remained high. This shift in the technological landscape played a crucial role in the evolution of the news analysis system.
During this period, the author worked for a small web shop that went bankrupt, leading to a period of job insecurity. However, after the web shop's collapse, the author found demand for white-hat search engine marketing expertise due to the Google quality rater job on their CV.
In a temporary role, the author worked as a Google quality rater, testing the search algorithm's results against human judgement. This experience further deepened the author's understanding of what constitutes good content, leading to the creation of software for analysing language around a given topic.
Eventually, the author discovered that their work in language analysis was part of the already-existing field of computational linguistics. This realisation underscored the author's unique contribution to the field, combining technical expertise with a deep understanding of media analysis.
In conclusion, the author's news corpus analysis system is a testament to the power of innovation and perseverance. From its humble beginnings to its current state, the system has provided valuable insights into the world of media and technology, and continues to evolve as the digital landscape changes.
The news analysis system, initially built on a laptop, was later transitioned to a Raspberry Pi and USB hard disk, illustrating how advancements in home-and-garden technology like the Raspberry Pi can be applied to lifestyle fields such as data-and-cloud-computing and media analysis. As the system evolved, it offered insights into trends and linguistic patterns, enriching the author's understanding of both technical aspects and the digital world.
