Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
spatialspecial [2017/04/19 16:39]
qcbs [Exercice 6]
spatialspecial [2019/04/15 16:58] (current)
qcbs [Exercice 7]
Line 1: Line 1:
 ====== QCBS workshop on the use of open source tools for advanced spatial analysis ====== ====== QCBS workshop on the use of open source tools for advanced spatial analysis ======
  
-April 20 2017, McGill University.  ​+April 16 2019, McGill University.  ​
  
   * [[https://​prezi.com/​z8ybqj4wxixp/​qcbs-workshop-on-the-use-of-open-source-tools-for-advanced-spatial-analyses/​|Link to the Prezi presentation]]   * [[https://​prezi.com/​z8ybqj4wxixp/​qcbs-workshop-on-the-use-of-open-source-tools-for-advanced-spatial-analyses/​|Link to the Prezi presentation]]
Line 10: Line 10:
 For Windows and Mac users For Windows and Mac users
  
-  * Installez ​PostgreSQL ​depuis le site web [[http://​www.enterprisedb.com/​products-services-training/​pgdownload|EntrepriseDB]] +  * Install ​PostgreSQL ​from the EntrepriseDB website ​[[http://​www.enterprisedb.com/​products-services-training/​pgdownload|EntrepriseDB]] 
-  * Quand on vous demande si vous voulez utiliser stackbuilderacceptez et choisissez d'​installer  ​PostGIS ​sous "​spatial extensions"​. ​Quand on vous le demandechoisissez de créer une base de données spatiale et nommez-là spatialdbRépondez oui aux questions ​à propos de  ​GDAL_DATA, POSTGIS_GDAL_ENABLED_DRIVERS,​ POSTGIS_ENABLED_OUTDB_RASTERS.+  * When asked if you want to install Stackbuilderchoose yes and select to install ​PostGIS ​under "​spatial extensions"​. ​When you are askedchoose to create a new spatial database and name it spatiadbAnswer yes to questions ​about GDAL_DATA, POSTGIS_GDAL_ENABLED_DRIVERS,​ POSTGIS_ENABLED_OUTDB_RASTERS.
  
  
Line 21: Line 21:
 Then type Then type
 <​file>​ <​file>​
-sudo gedit /​etc/​postgresql/​9.6/​main/​postgresql.conf+sudo gedit /​etc/​postgresql/​11.2/​main/​postgresql.conf
 </​file>​ </​file>​
 remove the # in front of #​listen_addresses = '​localhost'​ remove the # in front of #​listen_addresses = '​localhost'​
Line 29: Line 29:
 sudo service postgresql restart sudo service postgresql restart
 </​file>​ </​file>​
 +
 +For Mac users. In the psql terminal, type:
 +<​file>​
 +SHOW hba_file;
 +</​file>​
 +
 +You then need to edit the postgresql.conf file to remove the # in front of #​listen_addresses = '​localhost'​. You can also Put Trust at the end of the lines with localhost. ​
  
 To create a spatial database To create a spatial database
Line 76: Line 83:
 **ST_Perimeter**(geometry) returns the length of all the rings\\ **ST_Perimeter**(geometry) returns the length of all the rings\\
 **ST_AsSVG**(geometry) returns SVG text\\ **ST_AsSVG**(geometry) returns SVG text\\
 +**ST_Transform**(geometry,​ SRID) change the reference system from the current one (of geom) to the one defined by SRID.\\
 +
 +Select all observations North of the 70th parallel
 +<file postgresql>​
 +SELECT * FROM obis_whales WHERE latitude > 70
 +</​file>​
 +
 +Select all observations South of the 60th parallel
 +<file postgresql>​
 +SELECT tname, latitude FROM obis_whales WHERE latitude < -60
 +</​file>​
 +
 +List the species that can be found South of the 60th parallel
 +<file postgresql>​
 +SELECT DISTINCT tname FROM obis_whales WHERE latitude < -60
 +</​file>​
  
 Create a new layer, with belugas only Create a new layer, with belugas only
Line 152: Line 175:
  
 ++++ To use a distance in meters | ++++ To use a distance in meters |
 +Add new empty column of type geography
 <file postgresql>​ <file postgresql>​
 ALTER TABLE obis_whales ADD column geog GEOGRAPHY(POINT,​4326) ALTER TABLE obis_whales ADD column geog GEOGRAPHY(POINT,​4326)
 </​file>​ </​file>​
  
 +Update the geometry values as geography and store them in the new column
 <file postgresql>​ <file postgresql>​
 UPDATE obis_whales SET geog=geography(geom) UPDATE obis_whales SET geog=geography(geom)
 </​file>​ </​file>​
  
 +Repeat for the tm_world file
 <file postgresql>​ <file postgresql>​
 ALTER TABLE tm_world ADD column geog GEOGRAPHY(MULTIPOLYGON,​4326) ALTER TABLE tm_world ADD column geog GEOGRAPHY(MULTIPOLYGON,​4326)
Line 212: Line 237:
 \\ \\
 \\ \\
-===== Using PostgreSQL/​PostGIS and GRASS in R =====+===== Using PostgreSQL/​PostGIS, GDAL, sf and GRASS in R =====
  
 {{::​spatial_special_r_code.r|Download the R file containing the code for this part of the workshop}} {{::​spatial_special_r_code.r|Download the R file containing the code for this part of the workshop}}
 +
  
   * Package [[http://​cran.r-project.org/​web/​packages/​rpostgis/​index.html|rpostgis]]   * Package [[http://​cran.r-project.org/​web/​packages/​rpostgis/​index.html|rpostgis]]
-  * Package [[http://cran.r-project.org/web/​packages/​rqgis/​index.html|rQGIS]] +  * Package [[https://github.com/jannes-m/RQGIS3|RQGIS3]] 
-  * Package [[http://​cran.r-project.org/​web/​packages/​spgrass6/​index.html|spgrass6]]+  * Package [[http://​cran.r-project.org/​web/​packages/​rgrass7/​index.html|rgrass7]]
   * Package [[http://​cran.r-project.org/​web/​packages/​RPostgreSQL/​index.html|RPostgreSQL]]   * Package [[http://​cran.r-project.org/​web/​packages/​RPostgreSQL/​index.html|RPostgreSQL]]
   * Package [[http://​cran.r-project.org/​web/​packages/​rgdal/​index.html|rgdal]]   * Package [[http://​cran.r-project.org/​web/​packages/​rgdal/​index.html|rgdal]]
Line 224: Line 250:
 ===== Using the Processing Framework in QGIS ===== ===== Using the Processing Framework in QGIS =====
  
-  ​[[http://​docs.qgis.org/​2.8/​en/​docs/​training_manual/​processing/​index.html|QGIS Training manual. ​Processing ​Guide. ]] +You first need to install the **Processing ​R Provider plugin** in QGISThen, restart ​QGIS
-  ​[[http://​docs.qgis.org/​2.8/​en/​docs/​user_manual/​processing/​index.html|QGIS User manual - Processing framework.]]+
  
 +  * [[https://​docs.qgis.org/​3.4/​en/​docs/​training_manual/​processing/​r_syntax.html|R Syntax in Processing scripts. ]]
  
 **Commands to specify the input and output files and script options in Processing** **Commands to specify the input and output files and script options in Processing**
Line 241: Line 267:
 **folder**. A folder.\\ **folder**. A folder.\\
 **file**. A filename.\\ **file**. A filename.\\
 +
 +To create a new R Processing script, click on the R logo on the top of the processing toolbox pane, and select Create New R Script. When you are done typing the script, give it an appropriate name and save it in the default folder that opens. ​
 ===== Exercice 5 ===== ===== Exercice 5 =====
-Open the Processing Toolbox. Make sure that R is activated as a provider in the Toolbox options. Find "R Scripts > Create a new R script" ​in the toolbox. Copy and past this Processing/​R script that creates 100 points distributed randomly on the earth in the latitude/​longitude reference system (EPSG 4326) and that are imported as a QGIS layer. ​+Create a new R script in the toolbox ​by copying ​and pasting ​this Processing/​R script that creates 100 points distributed randomly on the earth in the latitude/​longitude reference system (EPSG 4326) and that are imported as a QGIS layer. ​
  
 Overlay the result on top of the TM_World_Borders shapefile. ​ Overlay the result on top of the TM_World_Borders shapefile. ​
Line 269: Line 297:
 c2<​-st_as_sf(countries) c2<​-st_as_sf(countries)
 areas<​-st_area(c2) areas<​-st_area(c2)
-plot(log(areas),​log(c2$POP2005),​xlab='​Log Area',​ylab='​Log Population'​) +plot(log(areas),​log(as.numeric(c2$POP2005)),​xlab='​Log Area',​ylab='​Log Population'​) 
-text(log(areas),​log(c2$POP2005),​c2$NAME,​cex=0.7)+text(log(areas),​log(as.numeric(c2$POP2005)),​c2$NAME,​cex=0.7)
 </​file>​ </​file>​
 ++++ ++++
Line 284: Line 312:
 ##​Layer=vector ##​Layer=vector
 ##showplots ##showplots
-data2=tapply(Layer$new_area,​Layer$IHO_Sea,​mean)+#data2=tapply(Layer$new_area,​Layer$IHO_Sea,​mean
 +library('​dplyr'​) 
 +Layer=data.frame(Layer) 
 +data2=Layer%>​%group_by(IHO_Sea)%>​%summarize(mean=mean(new_area))
 par(mar=c(10,​4,​4,​2)) par(mar=c(10,​4,​4,​2))
-barplot(data2,​ las=2)+barplot(data2$mean, names.arg=data2$IHO_Sea, las=2)
 </​file>​ </​file>​
  
Line 293: Line 324:
 4 - Specify "​Country seas" as the name of the model and "QCBS Workshop"​ as the group. 4 - Specify "​Country seas" as the name of the model and "QCBS Workshop"​ as the group.
  
-5 - Add a vector layer to the model, name it "​Countries"​ and specify polygon as the "shape type"​. ​+5 - Add a vector layer to the model, name it "​Countries"​ and specify polygon as the "geometry ​type"​. ​
  
 6 - Add a "​String"​ to the model and name it, "​Country Name". 6 - Add a "​String"​ to the model and name it, "​Country Name".
  
-7 - Add "​Extract by attributes"​ to the model (under ​QGIS geoalgorithms). Specify "Country" as the "Input Layer",​ = as the operator, NAME as the "​Selection Attribute"​ and "​Country Name" as the "​Value"​.+7 - Add "​Extract by attributes"​ to the model (under ​Algorithms>​Vector selection). Specify "Countries" as the "Input Layer",​ = as the operator, NAME as the "​Selection Attribute"​, click on the Gear icon under Value, choose Model input and select ​"​Country Name" as the "​Value"​.
  
-8 - Add a "Fixed Distance ​Buffer"​ to the model. Specify "​Output from Extract by Attributes"​ as the "​Input",​ and specify the distance as 0.5 (it's in degrees since the map is in latitude/​longitude). Choose to "​dissolve the output"​.+8 - Add a "​Buffer"​ to the model. Specify "​Output from Extract by Attributes"​ as the "​Input",​ and specify the distance as 0.5 (it's in degrees since the map is in latitude/​longitude). Choose to "​dissolve the output" ​and leave other field blank.
  
-9 - Add another ​vector ​to the model. Name it "​Zones"​. Specify "​Polygon"​ as the "shape type". This will be our polygons layer containing the different seas. +9 - Add another ​Vector Layer to the model. Name it "Sea Zones"​. Specify "​Polygon"​ as the "shape type". This will be our polygons layer containing the different seas. 
  
-10 - Add "​Intersection"​ to the model. Specify an intersection between "output of the Buffer"​ and the layer "​Zones"​.+10 - Add "​Intersection"​ to the model. Specify an intersection between "Buffered from Buffer"​ and the layer "Sea Zones". Leave other field blank.
  
-11 - Add "Field Calculator"​ to the model. Specify new_area as "​Result Field Name" and $area as the fomula. This will calculate the area of each polygon of each section of sea in the buffer zone around the country in degrees squared.  ​+11 - Add "Field Calculator"​ to the model. Choose the intersection layer from the previous step as the input. Specify new_area as "​Result Field Name" and $area as the fomula. This will calculate the area of each polygon of each section of sea in the buffer zone around the country in degrees squared.  ​
  
 12 - Now, add the R script created at Step 2 to the model. "​Input"​ is the output from Step 11 (Field Calculator). Call the output file "​Country_seas"​ 12 - Now, add the R script created at Step 2 to the model. "​Input"​ is the output from Step 11 (Field Calculator). Call the output file "​Country_seas"​
Line 311: Line 342:
 Your model should look like this one:  Your model should look like this one: 
  
-{{::screenshot_from_2017-04-19_11-54-37.png?​direct&​200|}}+{{::screenshot_from_2019-04-15_16-51-00.png?​direct&​200|}}
  
 13 - You are now ready to run the model! First, save the model and name it Country_seas. Then, close the "​Graphical Modeler"​ and double-click on your model in the Toolbox under Models. Specify TM_World_Borders as the Country layer and EEZ_IHO_union_v2 for the zones. Specify the name of a country of your choice (ex. Canada). 13 - You are now ready to run the model! First, save the model and name it Country_seas. Then, close the "​Graphical Modeler"​ and double-click on your model in the Toolbox under Models. Specify TM_World_Borders as the Country layer and EEZ_IHO_union_v2 for the zones. Specify the name of a country of your choice (ex. Canada).