<\/p>\n
Data Acquisition<\/b><\/p>\n
Weather data was sourced from various DoseNet sensors (Figure 1), paired with nearby sensors on WeatherUnderground\u2019s network (weatherunderground.com). Due to long element load times on the website, Selenium webdriver\u2019s were used to extract the element containing the weather data. The data was then saved into an html file that was reformatted into a Pandas DataFrame through the <\/span>read_html<\/span><\/i> method. Using Pandas\u2019 built in <\/span>str.replace<\/span><\/i>, .<\/span>astype, and .rename<\/span><\/i>, I was able to clear the DataFrame of typographical symbols. This data was ready to be put into a linear regression model and used in calculations.<\/span><\/p>\n \u00a0<\/b><\/p>\n DoseNet sensors (i.e. Etcheverry Roof) gather data at five minute intervals, often deviating 1-2 minutes from WeatherUnderground\u2019s readings. This was often acceptable for simple tasks such as comparing readings for temperature between the two sensors. For more complex tasks (i.e. comparing precipitation or Csl data) it was necessary to have well-formatted data. Binning the data using Pandas resampling (for our analysis, we used 5 minutes) allowed for equal time distributions which fulfilled this requirement.\u00a0<\/span><\/p>\n Analysis and Results<\/b><\/p>\n To analyze the data gathered, I had to determine the best statistical measures for the limited information available and the distinct groups of data. For the purpose of extracting correlations, I found a linear regression model to work best. Paired with a t-test, I would have a good idea of the relative correlation between any two compared variables. My experiments started with the comparison readings between the \u2018Etcheverry Roof\u2019 and \u2018NearEtch\u2019 temperature sensors (Figure 1). When displaying the data sets in a linear regression model using Sci-Kit Learn\u2019s <\/span>LinearRegressor <\/span><\/i>function, it was apparent that there was a slight variation between the two datasets (Figure 2a).\u00a0 The t-test result, \u2018T-value\u2019, indicated that there was no statistically significant difference between the datasets (Figure 3, Figure 4), further supporting that there are only slight variations between the measurements. Similar results were seen for humidity and pressure measurements (Figure 2b, Figure 2c).<\/span><\/p>\n <\/p>\n <\/p>\n With the null hypothesis as \u2018no difference\u2019, the calculated p-values (Figure 4), all significantly higher than 0.05, indicate a very strong correlation between the DoseNet and WeatherUnderground temperature\/humidity\/pressure sensors.<\/span><\/p>\n <\/p>\n It is reasonable to assume any variations found could, in part, be due to differences in sensor conditions. These results were reflected in the rest of the sensor comparisons.\u00a0<\/span><\/p>\n <\/p>\n Using this template, I tackled unusual readings from precipitation (Decrisp) sensors at LBL. The sensor, working on a scale of 0-10, reported readings with no clear source. Initially, I compared the precipitation readings to precipitation data from WeatherUnderground, but no clear variation in the data was found. Any spikes were drowned out by \u2018zero\u2019 readings (Figure 5). No t-value or p-value was taken into consideration due to the overwhelming amount of equivalent \u2018zero\u2019 data (this indication of correlation would not be helpful.) Similar was found when comparing raw data, as WeatherUnderground\u2019s sensors reported no accumulation of precipitation.<\/span><\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/Screen-Shot-2022-09-14-at-1.10.05-AM.png\")
<\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_2a_Etch_Temp_06_08.png\")
![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_2b_Etch_Humidity_06_08.png\")
![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_2c_Etch_Pressure_06_08.png\")
<\/b><\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_3_tval-e1663144013196.png\")
<\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/Findings-4-e1663228362632.png\")
<\/p>\n
![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_5_LBL_NEARLBL_RAIN.png\")
<\/span><\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_6_LBL_NEARLBL_SPEED.png\")
<\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_6b_LBL_NEARLBL_SPEED-e1663227330903.png\")
<\/p>\n![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_7a_WU_ACCURACY_193_187_060822.png\")
![\"\"](\"https:\/\/radwatch.berkeley.edu\/wp-content\/uploads\/2022\/09\/figure_7b_WU_ACCURACY_HIST_060822_193_187.png\")
<\/p>\n