docs and tests

docs/source/index.rst

@@ -4,6 +4,11 @@ iDAS Convert
 Convert and downsample distribute acousting sensing data (DAS) acquired by `Silixa iDAS <https://silixa.com/products/idas/>`_ to seismological data formats.
 Main purpose is to quickly convert and downsample massive amounts of high-resolution TDMS data to MiniSeed.
 
+To handle the massive amount of data generated by DAS interogators, the conversion tool is leveraging parallel IO and parallel processing.
+On production systems a throughput of 200 MB/s while converting and downsampling (1 kHz -> 200 Hz) has been archived.
+
+The seismological signal processing routines are based on the `Pyrocko <https://pyrocko.org>`_ framework, are mature and well tested.
+
 Contents
 --------
 

docs/source/setup.rst

@@ -8,6 +8,7 @@ Python 3 requirements:
 
 * `pyrocko <https://pyrocko.org>`_
 * telebot (optional)
+* `numpy <https://numpy.org>`_
 
 Installation using pip
 
@@ -51,7 +52,7 @@ In case of an abort or error the conversion can be resumed with:
 Configuration
 -------------
 
-The conversion tools are configured with a `YAML <https://en.wikipedia.org/wiki/YAML>`_ file. This file is then **executed** by ``idas_convert`` CLI programm.
+The conversion tools are configured with a `YAML <https://en.wikipedia.org/wiki/YAML>`_ file. This file is then _executed_ by ``idas_convert`` CLI programm.
 
 .. code-block :: YAML
 
@@ -98,8 +99,11 @@ The conversion tools are configured with a `YAML <https://en.wikipedia.org/wiki/
     # A Telegram bot to keep up-to-date with the process
     - !idas_convert.telegram_bot.TelegramBotConfig
     enabled: false
-    token: Telegram Token
-    chat_id: Telegram Chat ID, e.g. -456413218
+    # Telegram API Token
+    token: 9e98b8c0567149eb861838a1d770be7d
+    # Telegram Chat ID
+    chat_id: -1237123123
+    # A status message will be dispatched every 3600 s
     status_interval: 3600.0
 
 Plugins
@@ -107,7 +111,7 @@ Plugins
 
 The following plugins can be configured in the ``plugins`` list in the YAML file.
 
-Telegram bot
+Telegram Bot
 ^^^^^^^^^^^^
 
 A Telegram bot can be configured to keep up-to-date with the processing progress.
@@ -126,7 +130,7 @@ Details about the Telegram ``token`` and ``chat_id`` can be found `here <https:/
     status_interval: 3600.0
 
 
-GFZ Tape interaction
+GFZ Tape Interaction
 ^^^^^^^^^^^^^^^^^^^^
 
 The `GFZ German Research Centre for Geosciences <https://gfz-potsdam.de>`_ maintains a tape storage system, details about the system `here <https://www.golem.de/news/bandlaufwerke-als-backupmedium-ein-bisschen-tetris-spielen-1906-141575.html>`_ (in German).

requirements.txt

@@ -1,2 +1,3 @@
+numpy
 pyrocko
 telebot

src/trace_stats.py

@@ -1,10 +0,0 @@
-import os
-import time
-import logging
-import subprocess
-
-from pyrocko.guts import Bool, Float
-from .plugin import Plugin, PluginConfig, register_plugin
-from .meta import Path, DataSize
-
-logger = logging.getLogger(__name__)

test/test_convert.py

@@ -0,0 +1,42 @@
+import numpy as num
+import pytest
+
+from hypothesis import given
+from hypothesis.strategies import floats, integers
+from hypothesis.extra.numpy import arrays
+
+
+from pyrocko.trace import Trace
+
+from idas_convert.idas_convert import split, process_data
+
+NSAMPLES = 1000
+
+
+@given(arrays(num.int32, 1000, elements=integers(-1000, 1000)))
+def test_split(data):
+    tr = Trace(
+        ydata=data,
+        deltat=0.01)
+    tmin_half = (tr.tmax - tr.tmin) / 2
+    traces = split(tr, tmin_half)
+
+    assert len(traces) == 2
+    assert sum(t.ydata.size for t in traces) == tr.ydata.size
+    assert traces[0].tmin == tr.tmin
+    assert traces[1].tmax == tr.tmax
+
+
+@given(arrays(num.int32, 10000, elements=integers(-1000, 1000)))
+def test_process_data(data):
+    tr = Trace(
+        ydata=data,
+        deltat=0.001)
+    
+    tmin = tr.tmin + .1
+    tmax = tr.tmax - .1
+    deltat = 0.005
+    chunk = (tr, deltat, tmin, tmax)
+    ptr = process_data(chunk)
+    assert ptr.deltat == deltat
+

test/test_various.py

@@ -0,0 +1,17 @@
+from pyrocko.guts import Object
+
+from idas_convert.meta import DataSize
+from idas_convert.plugin import Plugin
+
+def test_data_size():
+
+    class Test(Object):
+        data_size = DataSize.T()
+    
+    t = Test(data_size=10000)
+    assert t.data_size == 10000
+
+
+def test_plugin_base():
+    p = Plugin()
+    p.set_parent('123')