Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,27 @@ with mcstastox.Read(file_path) as loaded_data:
```

This takes less space and events are already grouped by pixel ids

Weighted McStas events can be converted to normal unit-weight events while
exporting by passing sampling settings. Sampling is performed as a stream, so
only the requested number of events is kept in memory:

```
import mcstastox

settings = mcstastox.SamplingSettings(n_samples=100_000, seed=42)
with mcstastox.Read(file_path) as loaded_data:
sampled = loaded_data.export_scipp(
source_name="source",
sample_name="sample_position",
sampling=settings,
)
```

Sampling is weighted by the original event probabilities, allows repeated
events, and sets every sampled event weight to one. Set `ordered=True` to keep
the order of the input event stream. The sampled scipp object also contains a
scalar `effective_duration` coordinate. When the input weights are detector
count rates, this is `n_samples / sum(weight)` in seconds. The input weight
column defaults to `p` and can be changed with `weight_key`. Its Scipp
variance is propagated from the McStas weight variance `sum(weight**2)`.
1 change: 1 addition & 0 deletions docs/user-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ maxdepth: 1
installation
overall_demo.ipynb
scipp_demo.ipynb
stream_sampling_demo.ipynb
```
4 changes: 3 additions & 1 deletion docs/user-guide/read_example.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
import os

import matplotlib.pyplot as plt
import mcstasscript as ms


def make_instrument(
square=True, banana=True, id_overlap=False, id_gap=False, non_zero_id_start=False
):
instr = ms.McStas_instr("test")
instr = ms.McStas_instr(f"test_{os.getpid()}")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to make the instrument unique?
Then we can use uuid instead maybe.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since multiple documentation notebooks build that instrument and execute it in parallel, it is to avoid the data files being named the same. In sequence, it is detected and avoided, but not in parallel.

I think this problem can be solved in a nicer way by only making the data folder name funny, not also the instrument name. Would be nice to merge and get it in a later round!

source = instr.add_component("source", "Source_simple")
source.set_parameters(
xwidth=0.01,
Expand Down
244 changes: 244 additions & 0 deletions docs/user-guide/stream_sampling_demo.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"# Weighted event sampling to `scipp`\n",
"\n",
"McStas event files contain weighted events. This notebook focuses on converting those weighted events to a fixed number of normal, unit-weight events while exporting to the simple `scipp` representation."
]
},
{
"cell_type": "markdown",
"id": "1",
"metadata": {},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"import numpy as np\n",
"\n",
"import mcstastox\n",
"from read_example import make_instrument"
]
},
{
"cell_type": "markdown",
"id": "3",
"metadata": {},
"source": [
"## Run a small McStas simulation\n",
"\n",
"As in the other user-guide notebooks, use `read_example` to create a small instrument and generate a McStas NeXus event file. The `Read` interface takes the output folder, while McStas writes the file as `mccode.h5` inside that folder."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4",
"metadata": {},
"outputs": [],
"source": [
"instr = make_instrument()\n",
"instr.set_parameters(wavelength=1.8, delta_wavelength=1.3)\n",
"instr.settings(ncount=1E6)\n",
"data = instr.backengine()\n",
"data_folder = Path(data[0].original_data_location)\n",
"data_folder"
]
},
{
"cell_type": "markdown",
"id": "5",
"metadata": {},
"source": [
"## Inspect the weighted `scipp_simple` export\n",
"\n",
"`export_scipp_simple` returns one `scipp.DataArray`. Before sampling, its data values are the McStas event weights (`p`). Reading in chunks avoids assembling the complete event file while it is loaded."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6",
"metadata": {},
"outputs": [],
"source": [
"chunk_size = 1_000\n",
"wavelength = mcstastox.Transfer(\n",
" mcstas_variable=\"L\",\n",
" scipp_coord=\"sim_wavelength\",\n",
" unit=\"angstrom\",\n",
")\n",
"\n",
"with mcstastox.Read(data_folder) as loaded_data:\n",
" weighted = loaded_data.export_scipp_simple(\n",
" source_name=\"source\",\n",
" sample_name=\"sample_position\",\n",
" component_name=\"Square_1\",\n",
" extra_variables=wavelength,\n",
" chunk_size=chunk_size,\n",
" )\n",
"\n",
"weighted"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7",
"metadata": {},
"outputs": [],
"source": [
"weighted.sizes, weighted.values[:5], weighted.coords[\"sim_wavelength\"].values[:5]"
]
},
{
"cell_type": "markdown",
"id": "8",
"metadata": {},
"source": [
"## Configure sampling\n",
"\n",
"`SamplingSettings` uses weighted reservoir sampling with replacement. The result contains exactly `n_samples` events, so an input event may appear more than once. A seed makes the result reproducible. The weight column defaults to `p`; use `weight_key` when the input stores weights under another name.\n",
"\n",
"Here, the **input stream** is the sequence of event records read from the selected McStas component. Each record contains the configured weight column (`p` by default), time (`t`), pixel ID (`id`), and any requested extra variables. With chunked loading, the stream is the same logical sequence as a full read: rows are visited in file order, and selected components are visited in component order.\n",
"\n",
"By default, the reservoir is returned in its internal sampling order, so the output order is not meaningful. Set `ordered=True` to retain each sampled record's position in the input stream and sort the result back into that order. This changes only the arrangement of the sampled records, not which weighted sample is selected."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9",
"metadata": {},
"outputs": [],
"source": [
"sampling = mcstastox.SamplingSettings(\n",
" n_samples=1_000,\n",
" seed=42,\n",
" ordered=True,\n",
")\n",
"sampling"
]
},
{
"cell_type": "markdown",
"id": "10",
"metadata": {},
"source": [
"## Export sampled normal events\n",
"\n",
"Pass the settings with the `sampling=` keyword. Sampling happens while the input chunks are read, and the exported data values are all one count. Extra event coordinates are sampled with the same events and remain aligned. The output also has a scalar `effective_duration` coordinate: when the configured weight column is a detector count rate, it is `n_samples / sum(weight)` seconds, with variance propagated from `sum(weight**2)`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "11",
"metadata": {},
"outputs": [],
"source": [
"with mcstastox.Read(data_folder) as loaded_data:\n",
" sampled = loaded_data.export_scipp_simple(\n",
" source_name=\"source\",\n",
" sample_name=\"sample_position\",\n",
" component_name=\"Square_1\",\n",
" extra_variables=wavelength,\n",
" chunk_size=chunk_size,\n",
" sampling=sampling,\n",
" )\n",
"\n",
"sampled"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12",
"metadata": {},
"outputs": [],
"source": [
"assert sampled.sizes[\"events\"] == sampling.n_samples\n",
"np.testing.assert_array_equal(sampled.values, np.ones(sampling.n_samples))\n",
"sampled.coords[\"effective_duration\"]\n",
"\n",
"sampled.sizes, sampled.values[:5], sampled.coords[\"sim_wavelength\"].values[:5]"
]
},
{
"cell_type": "markdown",
"id": "13",
"metadata": {},
"source": [
"The same settings can be used to reproduce the same event selection. Change `seed` to obtain another statistically weighted sample."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14",
"metadata": {},
"outputs": [],
"source": [
"with mcstastox.Read(data_folder) as loaded_data:\n",
" sampled_again = loaded_data.export_scipp_simple(\n",
" source_name=\"source\",\n",
" sample_name=\"sample_position\",\n",
" component_name=\"Square_1\",\n",
" extra_variables=wavelength,\n",
" chunk_size=chunk_size,\n",
" sampling=sampling,\n",
" )\n",
"\n",
"np.testing.assert_array_equal(sampled.values, sampled_again.values)\n",
"np.testing.assert_array_equal(\n",
" sampled.coords[\"t\"].values, sampled_again.coords[\"t\"].values\n",
")\n",
"print(\"Sampling is reproducible with the same seed.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "15",
"metadata": {},
"outputs": [],
"source": [
"sampled"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.13"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading
Loading