{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "global-alert",
   "metadata": {},
   "source": [
    "# Sources and Measurements Example\n",
    "\n",
    "This example shows how to upload measurements from an external repository into the MultiViz Analytics Engine (MVG) service and how to read this data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "medical-platform",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:16.678492Z",
     "start_time": "2021-03-18T09:54:16.487989Z"
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "from pathlib import Path\n",
    "\n",
    "from tqdm.notebook import tqdm\n",
    "import pandas as pd\n",
    "\n",
    "# import mvg library with python bindings to mvg-API\n",
    "from mvg import MVG\n",
    "from mvg.exceptions import MVGAPIError"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "commercial-confidentiality",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-04T14:04:45.710183Z",
     "start_time": "2021-03-04T14:04:45.705726Z"
    }
   },
   "source": [
    ".. note:: Each `TOKEN` is used both for authorization and authentication.\n",
    "Thus, each unique token represents a unique user and each user has their own unique database on the VA-MVG' service.\n",
    "\n",
    "**You need to insert your token received from Viking Analytics here:**\n",
    "Just replace `\"os.environ['TEST_TOKEN']\"` by your token as a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "finished-emphasis",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:16.681622Z",
     "start_time": "2021-03-18T09:54:16.679684Z"
    },
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "ENDPOINT = \"https://api.beta.multiviz.com\"\n",
    "# Replace by your own Token\n",
    "VALID_TOKEN = os.environ['TEST_TOKEN']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prompt-concentrate",
   "metadata": {},
   "source": [
    "### Downloading the Data\n",
    "\n",
    "We pick the data from one source from our public charlie repo https://github.com/vikinganalytics/va-data-charlie.git for convenience.\n",
    "Clone that repository to get access to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cutting-entry",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:16.800754Z",
     "start_time": "2021-03-18T09:54:16.689464Z"
    }
   },
   "outputs": [],
   "source": [
    "!git clone https://github.com/vikinganalytics/va-data-charlie.git"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prostate-condition",
   "metadata": {},
   "source": [
    "We are going to use six of the sources from the charlie dataset with IDs `u0001` to `u0006`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "tracked-citizenship",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:16.804606Z",
     "start_time": "2021-03-18T09:54:16.802048Z"
    }
   },
   "outputs": [],
   "source": [
    "REF_DB_PATH = Path.cwd() / \"va-data-charlie\" / \"charlieDb\" / \"acc\"\n",
    "SOURCE_IDS = [\"u0001\", \"u0002\", \"u0003\", \"u0004\", \"u0005\", \"u0006\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "third-cable",
   "metadata": {},
   "source": [
    "## Connect to the API\n",
    "\n",
    "Instantiate a session object with MVG library.\n",
    "A session object basically catches the endpoint and the token,\n",
    "to simplify the calls to the MVG library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "humanitarian-listing",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:16.957584Z",
     "start_time": "2021-03-18T09:54:16.805695Z"
    }
   },
   "outputs": [],
   "source": [
    "session = MVG(ENDPOINT, VALID_TOKEN)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "secret-turning",
   "metadata": {},
   "source": [
    "We now check if the server is alive. The hello message contains, amongst others the API version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "legitimate-nothing",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.111625Z",
     "start_time": "2021-03-18T09:54:16.958633Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "{'name': 'MultiViz Engine API',\n 'version': 'v0.3.2',\n 'swagger': 'http://api.beta.multiviz.com/docs'}"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session.say_hello()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "immune-appreciation",
   "metadata": {},
   "source": [
    "## Check Database\n",
    "\n",
    "We start by seeing if there are any sources in the database.\n",
    "A source represents a measurement source e.g. a measurement sensor.\n",
    "Note that sources can be used to represent a sensor with several channels.\n",
    "\n",
    "Because we want to start at a clean slate we will list all sources (potentially none) and delete all of them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "judicial-mineral",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.247583Z",
     "start_time": "2021-03-18T09:54:17.113206Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source info retrieved:\n",
      "{'source_id': 'u0001', 'meta': {'assetId': 'assetJ', 'measPoint': 'mloc01', 'location': 'cancun', 'timezone': 'Europe/Stockholm'}, 'properties': {'data_class': 'waveform', 'channels': ['acc']}}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "sources = session.list_sources()\n",
    "\n",
    "for src in sources:\n",
    "    # While the list returned contains all information\n",
    "    # about all known sources, it is also possible\n",
    "    # to query for a single source by its id\n",
    "    s_info = session.get_source(src['source_id'])\n",
    "    print(f\"Source info retrieved:\\n{s_info}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "collectible-shame",
   "metadata": {},
   "source": [
    "The example below revolves around a source with source id u0001"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "annoying-president",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.250783Z",
     "start_time": "2021-03-18T09:54:17.248845Z"
    }
   },
   "outputs": [],
   "source": [
    "SOURCE_ID = SOURCE_IDS[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "voluntary-traveler",
   "metadata": {},
   "source": [
    "To make sure we start from a clean slate we delete our source in case it exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "pharmaceutical-producer",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.408675Z",
     "start_time": "2021-03-18T09:54:17.252056Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source u0001 deleted\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    source = session.get_source(SOURCE_ID)\n",
    "    session.delete_source(SOURCE_ID)\n",
    "    print(f\"Source {SOURCE_ID} deleted\")\n",
    "except MVGAPIError:\n",
    "    print(f\"Source {SOURCE_ID} does not exist\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "operating-identifier",
   "metadata": {},
   "source": [
    "## Build measurements\n",
    "\n",
    "Now we want to (re) build the source and the attached measurements from scratch.\n",
    "In this example, we have a json file with all the information needed to create each source."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "distant-institute",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.413080Z",
     "start_time": "2021-03-18T09:54:17.409985Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating meta info\n",
      "{'assetId': 'assetA', 'measPoint': 'mloc01', 'location': 'paris'}\n"
     ]
    }
   ],
   "source": [
    "src_path = REF_DB_PATH / SOURCE_ID\n",
    "m_file_name = REF_DB_PATH / SOURCE_ID / \"meta.json\"\n",
    "with open(m_file_name, \"r\") as json_file:\n",
    "    meta = json.load(json_file)\n",
    "print(\"Creating meta info\")\n",
    "print(meta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "shaped-enough",
   "metadata": {},
   "source": [
    "Create the source and check for it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "charitable-gnome",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.688070Z",
     "start_time": "2021-03-18T09:54:17.414133Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recreated source info:\n",
      "{'source_id': 'u0001', 'meta': {'assetId': 'assetA', 'measPoint': 'mloc01', 'location': 'paris'}, 'properties': {'data_class': 'waveform', 'channels': ['acc']}}\n"
     ]
    }
   ],
   "source": [
    "session.create_source(SOURCE_ID, channels=[\"acc\"], meta=meta)  # create\n",
    "source = session.get_source(SOURCE_ID)\n",
    "print(\"Recreated source info:\")\n",
    "print(source)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "macro-supplier",
   "metadata": {},
   "source": [
    "**Update** the source in case it is necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "consolidated-adventure",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.983670Z",
     "start_time": "2021-03-18T09:54:17.689219Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated source info\n",
      "{'source_id': 'u0001', 'meta': {'assetId': 'assetA', 'measPoint': 'mloc01', 'location': 'paris', 'updated': 'YES! I have been updated'}, 'properties': {'data_class': 'waveform', 'channels': ['acc']}}\n"
     ]
    }
   ],
   "source": [
    "meta['updated'] = \"YES! I have been updated\"\n",
    "session.update_source(SOURCE_ID, meta)  # update\n",
    "source = session.get_source(SOURCE_ID)\n",
    "print(\"Updated source info\")\n",
    "print(source)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dense-handling",
   "metadata": {},
   "source": [
    "Upload measurements to source. Measurements are tied to sources, they consist of\n",
    "\n",
    "- an array of floating point values with a header indicating the channel name\n",
    "- timestamp when the values were sampled\n",
    "- a field for the duration of the measurement\n",
    "- meta information to be stored along the measurement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "parliamentary-disco",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:54:17.986967Z",
     "start_time": "2021-03-18T09:54:17.984640Z"
    }
   },
   "outputs": [],
   "source": [
    "meas = [f.stem for f in Path(src_path).glob(\"*.csv\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tender-barrier",
   "metadata": {},
   "source": [
    "`meas` now contains a list of timestamps representing the measurements in our repo we upload from.\n",
    "We proceed to iterate over all of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "racial-dance",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:15.657580Z",
     "start_time": "2021-03-18T09:54:17.988134Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading measurements to source u0001\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/50 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "6daae2e2565b4aefa879c83640d15b96"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(f\"Uploading measurements to source {SOURCE_ID}\")\n",
    "for m in tqdm(meas):\n",
    "\n",
    "    # samples file for one measurement\n",
    "    TS_MEAS = str(m) + \".csv\"  # filename\n",
    "    TS_MEAS = REF_DB_PATH / SOURCE_ID / TS_MEAS  # path to file\n",
    "    ts_df = pd.read_csv(TS_MEAS)  # read csv into df\n",
    "    accs = ts_df.iloc[:, 0].tolist()  # convert to list\n",
    "\n",
    "    # meta information file for one measurement\n",
    "    TS_META = str(m) + \".json\"  # filename\n",
    "    TS_META = REF_DB_PATH / SOURCE_ID / TS_META  # path\n",
    "    with open(TS_META, \"r\") as json_file:  # read json\n",
    "        meas_info = json.load(json_file)  # into dict\n",
    "\n",
    "    # get duration and other meta info\n",
    "    duration = meas_info['duration']\n",
    "    meta_info = meas_info['meta']\n",
    "    # add sampling rate, not required by vibration API\n",
    "    # but may be used on client side\n",
    "    # in general any information can be stored\n",
    "    # along the actual samples\n",
    "    meta_info['sampling_rate'] = len(accs)/duration\n",
    "    # <<< end of code specific for repo\n",
    "\n",
    "    try:\n",
    "        # see mvg for details on this call\n",
    "        session.create_measurement(sid=SOURCE_ID,\n",
    "                                   duration=duration,\n",
    "                                   timestamp=int(m),\n",
    "                                   data={\"acc\": accs},\n",
    "                                   meta=meta_info)\n",
    "    except MVGAPIError as exc:\n",
    "        print(exc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "historic-variance",
   "metadata": {},
   "source": [
    "## Read the Measurements"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ahead-marketplace",
   "metadata": {},
   "source": [
    "Check if we actually created the measurements by reading them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "restricted-detroit",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:41.728688Z",
     "start_time": "2021-03-18T09:55:15.658652Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Read 50 stored measurements\n"
     ]
    }
   ],
   "source": [
    "m = session.list_measurements(SOURCE_ID)\n",
    "print(f\"Read {len(m)} stored measurements\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "paperback-yugoslavia",
   "metadata": {},
   "source": [
    "It is also possible to read a specific measurement at a single timestamp.\n",
    "Let's get the timestamp of the first measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "digital-stevens",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:42.225546Z",
     "start_time": "2021-03-18T09:55:41.729876Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "source_id: u0001\n",
      "timestamp: 1570186860\n",
      "duration:  2.8672073400507907\n",
      "meta:      {'sampling_rate': 13950.857142857141}\n",
      "data:      [0.63897705078125, -0.55078125]...\n"
     ]
    }
   ],
   "source": [
    "ts_0 = m[0]['timestamp']\n",
    "meas_0 = session.read_single_measurement(SOURCE_ID, ts_0)\n",
    "# we'll printout the returned measurement\n",
    "print(f\"source_id: {SOURCE_ID}\")\n",
    "print(f\"timestamp: {ts_0}\")\n",
    "print(f\"duration:  {meas_0['duration']}\")\n",
    "print(f\"meta:      {meas_0['meta']}\")\n",
    "print(f\"data:      {meas_0['data']['acc'][1:3]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mechanical-alfred",
   "metadata": {},
   "source": [
    "We can update the meta information for a measurement at a single timestamp.\n",
    "Let's get the timestamp of the first measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "impressed-brown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:42.870908Z",
     "start_time": "2021-03-18T09:55:42.226689Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "source_id: u0001\n",
      "timestamp: 1570186860\n",
      "duration:  2.8672073400507907\n",
      "meta:      {'sampling_rate': 13950.857142857141, 'updated': 'YES!'}\n",
      "data:      [0.63897705078125, -0.55078125]...\n"
     ]
    }
   ],
   "source": [
    "ts_0 = m[0]['timestamp']\n",
    "new_meta = meas_0['meta']\n",
    "new_meta['updated'] = \"YES!\"\n",
    "session.update_measurement(SOURCE_ID, ts_0, new_meta)\n",
    "meas_0_u = session.read_single_measurement(SOURCE_ID, ts_0)\n",
    "\n",
    "# we'll printout the returned measurement\n",
    "print(f\"source_id: {SOURCE_ID}\")\n",
    "print(f\"timestamp: {ts_0}\")\n",
    "print(f\"duration:  {meas_0_u['duration']}\")\n",
    "print(f\"meta:      {meas_0_u['meta']}\")\n",
    "print(f\"data:      {meas_0_u['data']['acc'][1:3]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ethical-amplifier",
   "metadata": {},
   "source": [
    "Finally, we delete the measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "respective-thread",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:43.023114Z",
     "start_time": "2021-03-18T09:55:42.872176Z"
    }
   },
   "outputs": [],
   "source": [
    "session.delete_measurement(SOURCE_ID, ts_0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "royal-gambling",
   "metadata": {},
   "source": [
    "We check if it's actually deleted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "czech-spread",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:55:43.153589Z",
     "start_time": "2021-03-18T09:55:43.024498Z"
    }
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    meas_0 = session.read_single_measurement(SOURCE_ID, ts_0)\n",
    "except MVGAPIError:\n",
    "    print(\"Previously deleted measurement does not exist\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aggressive-copying",
   "metadata": {},
   "source": [
    "If all went well we end up here now.\n",
    "The source u0001 is in the database along with a number of its measurements."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "animal-background",
   "metadata": {},
   "source": [
    "## Uploading additional sources\n",
    "\n",
    "Here we show how to upload additional sources in a more compact way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "working-content",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-18T09:56:43.305948Z",
     "start_time": "2021-03-18T09:55:43.154900Z"
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading sources u0002, u0003, u0004, u0005, u0006 from CharlieDb\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/5 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "bdc97293850b4e4c97bb2b75dcf79bb2"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating source u0002\n",
      "Uploading measurements to source u0002\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/50 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "b7db68b9080d438b84ea6df3411be8f4"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating source u0003\n",
      "Uploading measurements to source u0003\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/50 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "8f301abe49564c6594e8bf503a2bf7c0"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating source u0004\n",
      "Uploading measurements to source u0004\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/50 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "56ec663ca8754cdbac4cecadc82f8331"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating source u0005\n",
      "Uploading measurements to source u0005\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/50 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "2fbd54fb56b247ef9aa74368fd9b4a84"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating source u0006\n",
      "Uploading measurements to source u0006\n"
     ]
    },
    {
     "data": {
      "text/plain": "  0%|          | 0/42 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "f0544174950e4962a4c132408958a826"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sources = SOURCE_IDS[1:]\n",
    "\n",
    "print(f\"Uploading sources {', '.join(sources)} from CharlieDb\")\n",
    "for source_id in tqdm(sources):\n",
    "\n",
    "    try:\n",
    "        session.delete_source(source_id)\n",
    "    except MVGAPIError as exc:\n",
    "        pass  # Source didnt exist\n",
    "\n",
    "    print(f\"Creating source {source_id}\")\n",
    "    src_path = REF_DB_PATH / source_id\n",
    "    m_file_name = REF_DB_PATH / source_id / \"meta.json\"\n",
    "    with open(m_file_name, \"r\") as json_file:\n",
    "        meta = json.load(json_file)\n",
    "    session.create_source(source_id, channels=[\"acc\"], meta=meta)  # create\n",
    "\n",
    "    meas = [f.stem for f in Path(src_path).glob(\"*.csv\")]\n",
    "\n",
    "    print(f\"Uploading measurements to source {source_id}\")\n",
    "    for m in tqdm(meas):\n",
    "\n",
    "        # samples file for one measurement\n",
    "        TS_MEAS = str(m) + \".csv\"  # filename\n",
    "        TS_MEAS = REF_DB_PATH / source_id / TS_MEAS  # path to file\n",
    "        ts_df = pd.read_csv(TS_MEAS)  # read csv into df\n",
    "        accs = ts_df.iloc[:, 0].tolist()  # convert to list\n",
    "\n",
    "        # meta information file for one measurement\n",
    "        TS_META = str(m) + \".json\"  # filename\n",
    "        TS_META = REF_DB_PATH / source_id / TS_META  # path\n",
    "        with open(TS_META, \"r\") as json_file:  # read json\n",
    "            meas_info = json.load(json_file)  # into dict\n",
    "\n",
    "        duration = meas_info['duration']\n",
    "        meta_info = meas_info['meta']\n",
    "        meta_info['sampling_rate'] = len(accs)/duration\n",
    "\n",
    "        try:\n",
    "            # see mvg for details on this call\n",
    "            session.create_measurement(sid=source_id,\n",
    "                                       duration=duration,\n",
    "                                       timestamp=int(m),\n",
    "                                       data={\"acc\": accs},\n",
    "                                       meta=meta_info)\n",
    "        except MVGAPIError as exc:\n",
    "            print(exc)\n",
    "            raise\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "384px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}