Hi, this is recording the procedures for creating the file, the input file for the web
app developed by Sir
Coy and his team. And this is essentially; this is essentially the pre-processing of the raw data and
transforming it into the format needed by the web app. So this includes cleaning up the data as well as
running it through the model developed by Dr. Ed Calzeta. So this will be in two parts.
First is the data acquisition and checking. And second would be the FDRS model spreadsheet. So for the
first part, the data acquisition and checking will be divided into three parts. Part one will be checking for
completeness of data. Part two will be the cleanup and data column checking. And third will be the
preparation for model ingestion. So let's do the first part. The first part will be checking for
completeness of data. For this example, we'll be using the Region 2 weather data. Now, we are
expecting data formats, data formatted in various data that are differently formatted.
So in this case, we have columns for the date for the specific data and we have one two three four
columns of data this includes the cumulative specification maximum temperature relative humidity and
wind speed so if in case these columns are mixed up we are suggesting to rearrange them into the order
of these four parameters so also we also have to check the completeness of the number of days.
So supposedly this spreadsheet has 366, 365 or 66 rows of data but right now we only have 236. So we
have to note that this data is partially incomplete but it's okay since the model the spreadsheet that we
that we will use for creating the file takes this into consideration and fills in the missing days with no
data.
okay so we have to check that there are if there are dates that are present but has no data we have to
remove those dates for example we have missing data here What we have to do is to remove the entire
rows that has no data. Okay, so let's check it for completeness. And then the second part is to clean up
and check the data columns. Actually, we have already done that already.
We have reordered the columns into precipitation, temperature, humidity, and wind speed. So also we
have to carefully check the values. So for example, the values are in Fahrenheit, we have to convert that
to Celsius. So if the unit is not the same as this, then the values need to be converted.
Okay, so here we have some partial known data, but in this case I think we should put zeros on these
cells that has no data. Okay, actually it really doesn't matter if you put zeros or not because these next
spreadsheet we will take care of this missing data. Okay, so we have already checked the completeness
of the records.
We can now actually copy on this data and put them into the next spreadsheet so that will be the part
two of this two-step procedure
let's open up the fire index template currently it's version 1.52 and we have here three spreadsheets
within the workbook So first one is the data input, second is the output table, and third is the index
column. So for this section, we have four sub-steps.
The first one would be to add the cleaned up raw weather data. Second is to take care of the inter-year
model values. These are the values here.
inter-year model values
Third would be to select the appropriate municipality and fire model, fire index that will be exporting.
And Fourth would be to review the values of the actual model.
FIRST STEP
copy all this data, but as you can see, we have not included the headers, the headers of these columns
just copy that and paste them here as values. As you can see, the order of the parameters are like this.
So we have to make sure that in the raw data, they are ordered as the same. So when we paste these
values here, they would be reflected and they would correspond to their proper columns.
SECOND PART
So the second part would be the inter-year model values. As you can see here, this blue section here has
been updated with these values.
This data is for the year 2022. Now if we are going to process data for the year 2023, we have to take
note of these values and then write them onto this green section.
Then we will proceed to paste here the values for the 2023 parameters. What happens here is that the
model actually takes into consideration the value of the previous year in computing the index values for
the current year.
It's important that if you're processing values for 2022, we have to update this green section with the
values completed from this blue section. Okay, so that's it for the second part.
THIRD PART
Now, the third will be obvious. I'll do that change first. Okay. So, now that we have already entered the...
raw values. We can now go to the actual production of the csv format needed by the web app.
So as you can see here, there are 53 columns representing the 52 weeks of a year. And the values are
updated for the year for the year that you have pasted so it read that
it's 2022, so all the values will be 2022 - 1, 2, 3, 4, 5, 6, etc. weeks. So if we put in the 2023 data and
then this will be updated for the value of 2023
all right so currently this is black because we don't we have not yet selected the municipality that will be
producing the data for. For example, let's use latest from region 2.
So for example, let me check which places are there in region 2. Okay, so for example, let's use Cagayan
so you can select here Cagayan or you can actually start typing here and then it will figure out. Okay, so
now that we have selected the province of Cagayan, the spreadsheet automatically finds the
corresponding municipal ID. So this municipal ID also corresponds to the ID used by the web app.
So we will now select which index we are interested in producing. So actually we have to produce all six
for the province of Cagayan. So let's first define the fine fuel moisture index. So when we select that, the
values are actually now populated. So for the province of Cagayan, there are 29 municipalities. So what
we do now is to click Export CSV. Select all cells with data. Press Ctrl A here. So now all the data should
be highlighted.
Now we have a range of cells that will be exported as CSV. It will ask us where to put the file. So let's put
them here in the new folder. So let's call this 04 underscore output. Let's call this FFMI 2022 underscore
FMI underscore FMI. So the format is CSV. Let's check the folder.
And there we go. Let's also do this for the Duff Moisture Index. The index is updated. Select all.
and so on. So we do that for all the index values here.
fourth step
. And that's the fourth step, actually, this is just an appreciation of the values completed for each index
so click here.
And we see that it now has the it shows us the values computed for each individual index for each day of
the year okay so as you can see here it also has data only for 236 so that's the also the row number of all
the data available. So these are essentially the values for each of the day corresponding to the day
available
So now that we have outputted all the csv files. This will now be ready for importing or ingesting the
inputting to the webapp so that's it for the updated procedures for generating the files needed for the
webapp
Okay, this actually looks quite easy now because all the manual copy and pasting are already automated.
But if you want to see, the folders or the spreadsheets are just hidden so all the computations are here.
Let me check them out. So this spreadsheet controls the transposing and transforming the data into the
format needed by the web app. the individual days are written as rows, but they are summarized into
weekly values here.
And then the weekly values are transposed and written for each municipality so here the correction
factors for the land cover is applied. so we also have here the sheet for the correction factor so in case
that there's new land cover data available, then we will have to update these correction factors for the
land cover. As well as the anthropogenic, if we have additional data for
I think these correction factors are computed from the number of occurrences from the field data
collected and given to us by the vendor so as new data is gathered these correction factors will be
updated also So let's hide them here. And there you go. So that's it for the simplified procedures for
creating the CSV files needed by the web app.