April 8th, 2021

  • The complete data acquision on a dev server had been changed from DODS downloads to HTTPS downloads.

    (Speed) comparasion:
      DODS-downloads  HTTPS-downloads
    GFS-Parameters
    (atmoshere data)
     19  33 of more than 600 available
    Resolution in data points (lat*lon)  361*720  721*1440
    Timesteps  81 (3h, 10 days)  288 (1h, 5 days, 3h day 6-10)
    Datapoints 400 016 880 9 867 432 960
    Dataformat netcdf grib2
    Time to download  720 sec 170 sec
    speed factor per data point  104 times faster
         
    GFSwave-Parameters
    (ocean wave data)
     9  19
    Resolution (lat*lon)   336*720   721*1440
    Timesteps 61 (3h, 7,5 days)    288 (1h, 5 days, 3h day 6-10)
    Datapoints 132 814 080 5 681 249 280
    Dataformat netcdf grib2
    Time to download  360 sec 130 sec
    speed factor per data point  1 118 times faster 
         
    RTOFS-Parameters
    (Ocean data exept wave data)
     5  10
    Resolution (lat*lon)  2160*4320 3298*4500
    Timesteps  65 (3h, 8 days)  112 (1h, 3 days, 3h, day 4-8)
    Datapoints 3 032 640 000 16 621 920 000
    Dataformat netcdf netcdf
    Time to download  6240 sec 990 sec
    speed factor per data point  1 34 times faster 
         
  • new: parallel downloads (about 5-6 times faster with 10 parallel downloads)
  • new: partial https downloads are reducing the server load on NOAA side (about 5 times faster than requests through DODS server)
  • new: unsage o grib2 format instead of netcdf (about 4 times faster due to transmission of compressed data)
  • new. complete rewritten validation of correctness of the received data

 March 1st, 2021

  • Due to slow and unreliable downloads via the DODS-server, the decision for a change of the complete data acquision to https-downloads had been made.
    Also the support for the https downloads seems to be much better then for the DODS server.