3.2 KiB

Raw Permalink Blame History

DAS Convert

Convert and downsample distribute acoustic sensing (DAS) data acquired by Silixa iDAS to seismological data formats. Main purpose is to quickly convert and downsample massive amounts of high-resolution TDMS data to MiniSeed.

To handle the massive amount of data generated by DAS interrogators, the conversion tool is leveraging parallel IO and signal-processing processing. On production systems a throughput of 200 MB/s can be archived while converting and downsampling (from 1 kHz to 200 Hz).

The signal processing routines are based on the Pyrocko, mature and well tested seismological framework.

Installation

python3 setup.py install

Documentation

Find the online documentation at https://pyrocko.org/idas_convert/docs/.

Usage

Dump a config file

das_convert dump_config

Example Config

--- !idas.iDASConvertConfig
# Loading TDMS in parallel and process
nthreads_loading: 1
nthreads_processing: 8
queue_size: 32
processing_batch_size: 8

# Threads used for downsampling the data
nthreads: 8

# Input paths
paths:
- /home/isken/src/idas-convert

# Out path, see pyrocko.io for details
outpath: '%(tmin_year)s%(tmin_month)s%(tmin_day)s/%(network)s.%(station)s_%(tmin_year)s%(tmin_month)s%(tmin_day)s.mseed'

# Overwrite mseed meta information
new_network_code: ID
new_channel_code: HSF

downsample_to: 200.0

# MiniSeed record length
record_length: 4096
# MiniSeed STEIM compression
steim: 2

plugins:

# A plugin handling the communication with the GFZ tage file system
- !das_convert.gfz_tapes.GFZTapesConfig
  enabled: false
  bytes_stage: 1T
  waterlevel: 0.6
  wait_warning_interval: 600.0
  release_files: true
  path_tapes_mount: /projects/ether/
  path_tapes_prefix: /archive_FO1/RAW/

# A Telegram bot to keep up-to-date with the process
- !das_convert.telegram_bot.TelegramBotConfig
  enabled: false
  # Telegram API Token
  token: 9e98b8c0567149eb861838a1d770be7d
  # Telegram Chat ID
  chat_id: -1237123123
  # A status message will be dispatched every 3600 s
  status_interval: 3600.0

Start Conversion

das_convert my_config.yml

See das_convert -h for more options.

Citation

DAS Convert - Convert distributed acoustic sensing data, Isken, Marius Paul; Christopher, Wollin; Heimann, Sebastian; Javier, Quinteros; Karl-Heinz, Jäckel; Philippe, Jousset (2021): DAS Convert - Convert distributed acoustic sensing data. V. 1.0. GFZ Data Services. https://doi.org/10.5880/GFZ.2.1.2021.005

Legal Stuff

Licence: GNU General Public License, Version 3, 29 June 2007

Copyright © 2021 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany

DAS Convert is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. DAS Convert is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

3.2 KiB Raw Permalink Blame History