forked from admin/VolcanicAgenticCore
feat(google-calendar): encrypt chat tokens and give watchers a short-lived access token
token.json is sealed with AES-256-GCM (key derived from the OAuth client secret, bound to the chat), so the agent's processes can read the file but not the refresh token. Plain files, including VolcanicSecretary's refresh_token format, are rewritten sealed on first read. The scheduler gains IScriptEnvironment: extra per-run variables set after the process environment filter and redacted from the run's output. The Telegram host uses it to pass GOOGLE_ACCESS_TOKEN, refreshed to cover the job's timeout; the refresh token and client secret never reach the script.
This commit is contained in:
@@ -83,8 +83,8 @@ Read `docs/architecture.md` first. Key points:
|
||||
Extensions built on the public API only: providers `Anthropic` (Messages API), `OpenAI` (chat-completions compatible),
|
||||
`ClaudeCode` + `ClaudeCode.Bridge`; `Mcp` (an MCP client: stdio servers as tools, plus the Playwright browser
|
||||
preset; `docs/mcp.md`); the `GoogleCalendar` feature extension (`calendar` tool,
|
||||
`/calendar_connect|status|disconnect`, one Google account per chat keyed by session scope, tokens in the chat
|
||||
workspace under `.volcanic/google-calendar/`; `docs/google-calendar.md`); and the `Scheduler` feature extension (a process-wide
|
||||
`/calendar_connect|status|disconnect`, one Google account per chat keyed by session scope, tokens encrypted in the chat
|
||||
workspace under `.volcanic/google-calendar/`, watchers get a short-lived `GOOGLE_ACCESS_TOKEN`; `docs/google-calendar.md`); and the `Scheduler` feature extension (a process-wide
|
||||
`JobScheduler` on `TimeProvider`, `jobs.json` per chat in the workspace under `.volcanic/scheduler/`, the per-chat `schedule` tool, watcher
|
||||
scripts through `IProcessSpawnPolicy`; `docs/telegram-bot.md#scheduling`). The Claude Code provider is the most intricate piece
|
||||
(`docs/claude-code-provider.md`):
|
||||
|
||||
@@ -110,9 +110,15 @@ Each chat's refresh token is stored in its workspace, at `<workspace>/.volcanic/
|
||||
Telegram bot, `<WorkspaceRoot>/<chatId>/.volcanic/google-calendar/token.json`). It stays there across `/new`,
|
||||
resume and bot restarts.
|
||||
|
||||
- The file is encrypted: `{ "version": 2, "sealed": "..." }`, AES-256-GCM with a key derived from the OAuth client
|
||||
secret and bound to the chat. The agent can read the file with `read` or `bash` (it runs as the bot's OS user, and
|
||||
on some disks, such as NTFS, file modes mean nothing), but it never sees the client secret, so it can't decrypt
|
||||
the file. A file copied into another chat's folder doesn't open there either.
|
||||
- **A new client secret disconnects every chat**: their files no longer decrypt, and each chat runs
|
||||
`/calendar_connect` again.
|
||||
- A plain file, written before encryption or copied from VolcanicSecretary (`{ "refresh_token": "..." }`), is read
|
||||
once and rewritten encrypted.
|
||||
- On Unix the file has mode 0600 and its directory 0700.
|
||||
- The file is inside the workspace, so the agent can read it with `read` or `bash` and could send it with `send`.
|
||||
The refresh token gives full access to that chat's calendar; treat the workspace accordingly.
|
||||
- Deleting a chat's workspace disconnects its calendar locally. Run `/calendar_disconnect` first to also revoke the
|
||||
token at Google.
|
||||
|
||||
@@ -123,6 +129,22 @@ A host can keep tokens out of the workspaces with `GoogleCalendarOptions.TokenDi
|
||||
The file holds the refresh token, the granted scope, the account, the calendar's time zone and the connection time.
|
||||
The refresh token never appears in logs, tool results or error messages.
|
||||
|
||||
## Watcher scripts
|
||||
|
||||
In the Telegram bot, a chat's [watcher scripts](telegram-bot.md#scheduling) get `GOOGLE_ACCESS_TOKEN` when the chat
|
||||
has a connected calendar. The bot refreshes it before each run so that it is valid for the job's timeout plus a minute,
|
||||
up to Google's one-hour limit. A script calls the Calendar API with it directly:
|
||||
|
||||
```bash
|
||||
curl -s -H "Authorization: Bearer $GOOGLE_ACCESS_TOKEN" \
|
||||
"https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=$(date -u +%FT%TZ)&maxResults=5"
|
||||
```
|
||||
|
||||
The script never gets the refresh token or the client secret: if it leaks the access token, that token only works on
|
||||
this chat's calendar and expires within the hour. Its value is redacted from the script's output and failure alerts.
|
||||
Without a connected calendar, or when Google rejects the stored token, the variable is absent. Keep calendar watchers'
|
||||
timeouts short: a run that outlasts the token gets `401` from Google.
|
||||
|
||||
## Time zones
|
||||
|
||||
- `create` and `update` take an optional `timeZone` (an IANA id such as `Europe/Berlin`) for the event. Times without
|
||||
|
||||
@@ -27,7 +27,7 @@ target formats were checked against this repo's code.
|
||||
| `workspace/scripts/*` | `workspaces/<id>/scripts/` | Copy. The relative paths in job commands still resolve |
|
||||
| stray root files (reports, PDFs, …) | `workspaces/<id>/` | Copy |
|
||||
| `workspace/fortress/` profiles | — | **Drop** (we have no Fortress; Playwright profiles aren't compatible). Logins have to be redone |
|
||||
| `workspace/.secretary/google-token.json` | `workspaces/<id>/.volcanic/google-calendar/token.json` | **Convert** (§3.2) |
|
||||
| `workspace/.secretary/google-token.json` | `workspaces/<id>/.volcanic/google-calendar/token.json` | Copy; the bot encrypts it on first read (§3.2) |
|
||||
| `workspace/.secretary/jobs.json` | `workspaces/<id>/.volcanic/scheduler/jobs.json` | Copy; the scheduler converts it on start (§3.3) |
|
||||
| `pi-agent/sessions/--workspace--/*.jsonl` | `agent/sessions/telegram-<id>/<sessionId>.jsonl` | **Convert**; a verbatim copy won't load (§3.1) |
|
||||
| env `SECRETARY_TIMEZONE` (probably Europe/Warsaw) | `agent/telegram/chat-settings.json` | Generate `{"version":1,"chats":{"<id>":{"timeZone":"Europe/Warsaw"}}}` |
|
||||
@@ -79,10 +79,9 @@ with a fresh session that has a compaction summary built from the old one, inste
|
||||
|
||||
### 3.2 Google Calendar token
|
||||
|
||||
The source is `{"refresh_token":"1//0e…"}`. Our `GoogleTokenStore` reads `refreshToken`: key case is ignored but
|
||||
the underscore still matters, so a straight copy reads as "not connected". Write
|
||||
`{"refreshToken":"…","scope":"https://www.googleapis.com/auth/calendar","account":"<owner gmail>","timeZone":"Europe/Warsaw","connectedAt":"<now>"}`
|
||||
with mode 0600 and directories 0700.
|
||||
**Implemented:** `GoogleTokenStore` reads a plain `{"refresh_token":"1//0e…"}` and rewrites it encrypted
|
||||
(`{"version":2,"sealed":…}`, a key derived from the client secret, bound to the chat), so a straight copy works. The
|
||||
account and calendar time zone are unknown until a reconnect; the chat's zone is used meanwhile.
|
||||
|
||||
Both apps use the device flow with a "TVs and Limited Input devices" client. The refresh token is bound to the
|
||||
client, so **set our `GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET` to the Secretary's values**. With a different client,
|
||||
|
||||
@@ -164,7 +164,9 @@ script itself, **without the agent**:
|
||||
Later identical failures stay quiet until the script succeeds again.
|
||||
|
||||
Scripts run in the chat's workspace with the same filtered environment as `bash` and `TZ` set to the chat's zone.
|
||||
Known secrets are redacted from their output.
|
||||
When the chat has a connected Google Calendar they also get `GOOGLE_ACCESS_TOKEN`, a short-lived token for that
|
||||
calendar ([google-calendar.md](google-calendar.md#watcher-scripts)). Known secrets and that token are redacted from
|
||||
their output.
|
||||
|
||||
When a job fires while the agent is working, a scheduled prompt waits until the current work is done; it never
|
||||
interrupts. Occurrences that come due while the same job is still running are skipped.
|
||||
|
||||
@@ -28,7 +28,29 @@ public sealed class GoogleCalendarConnections
|
||||
var directory = Path.GetFullPath(settings.TokenRoot is { } root
|
||||
? Path.Combine(root, ChatKeys.For(scope))
|
||||
: Path.Combine(workspace, WorkspaceTokenDirectory));
|
||||
return _connections.GetOrAdd(directory, d => new GoogleCalendarConnection(settings, new GoogleTokenStore(Path.Combine(d, GoogleTokenStore.FileName))));
|
||||
return _connections.GetOrAdd(directory, d => new GoogleCalendarConnection(settings,
|
||||
new GoogleTokenStore(Path.Combine(d, GoogleTokenStore.FileName), new TokenCipher(settings.ClientSecret, ChatKeys.For(scope)))));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// For the host (e.g. a scheduled script's environment): the chat's access token, valid for at least
|
||||
/// <paramref name="validFor"/> as far as Google's token lifetime allows, or null when the calendar isn't configured or
|
||||
/// the chat isn't connected (or Google rejected its token). The refresh token and the client secret never leave this
|
||||
/// process. Other failures (network, Google errors) throw.
|
||||
/// </summary>
|
||||
public async Task<string?> TryGetAccessTokenAsync(GoogleCalendarOptions options, string scope, string workspace, TimeSpan validFor,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
if (options.Resolve() is not { } settings) return null;
|
||||
try
|
||||
{
|
||||
return await For(settings, scope, workspace).GetAccessTokenAsync(cancellationToken, validFor).ConfigureAwait(false);
|
||||
}
|
||||
catch (GoogleCalendarException ex) when (ex.NotConnected)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,18 +103,20 @@ internal sealed class GoogleCalendarConnection
|
||||
public void CancelPending() => Volatile.Read(ref _pending)?.Cancel();
|
||||
|
||||
/// <summary>
|
||||
/// A valid access token: the cached one while it has more than 60 s left and belongs to the stored refresh
|
||||
/// token, else one refresh shared by all concurrent callers.
|
||||
/// A valid access token: the cached one while it has more than 60 s (or <paramref name="validFor"/>, if longer) left
|
||||
/// and belongs to the stored refresh token, else one refresh shared by all concurrent callers. A fresh token is
|
||||
/// returned even when Google's lifetime is shorter than <paramref name="validFor"/>.
|
||||
/// </summary>
|
||||
public async Task<string> GetAccessTokenAsync(CancellationToken cancellationToken)
|
||||
public async Task<string> GetAccessTokenAsync(CancellationToken cancellationToken, TimeSpan? validFor = null)
|
||||
{
|
||||
var margin = validFor is { } wanted && wanted > RefreshMargin ? wanted : RefreshMargin;
|
||||
var stored = Store.Read() ?? throw new GoogleCalendarException(GoogleCalendarMessages.NotConnected, notConnected: true);
|
||||
if (Fresh(stored) is { } token) return token;
|
||||
if (Fresh(stored, margin) is { } token) return token;
|
||||
await _refreshLock.WaitAsync(cancellationToken).ConfigureAwait(false);
|
||||
try
|
||||
{
|
||||
stored = Store.Read() ?? throw new GoogleCalendarException(GoogleCalendarMessages.NotConnected, notConnected: true);
|
||||
if (Fresh(stored) is { } again) return again;
|
||||
if (Fresh(stored, margin) is { } again) return again;
|
||||
var access = await OAuth.RefreshAsync(stored.RefreshToken, cancellationToken).ConfigureAwait(false);
|
||||
Volatile.Write(ref _cached, new CachedToken(stored.RefreshToken, access.Token, access.ExpiresAt));
|
||||
return access.Token;
|
||||
@@ -119,9 +143,9 @@ internal sealed class GoogleCalendarConnection
|
||||
return chatZone;
|
||||
}
|
||||
|
||||
private string? Fresh(StoredToken stored) =>
|
||||
private string? Fresh(StoredToken stored, TimeSpan margin) =>
|
||||
Volatile.Read(ref _cached) is { } cached && cached.RefreshToken == stored.RefreshToken
|
||||
&& cached.ExpiresAt - Settings.Time.GetUtcNow() > RefreshMargin
|
||||
&& cached.ExpiresAt - Settings.Time.GetUtcNow() > margin
|
||||
? cached.AccessToken
|
||||
: null;
|
||||
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Nodes;
|
||||
using Volcanic.AgenticCore.Abstractions.Json;
|
||||
|
||||
namespace Volcanic.AgenticCore.Extensions.GoogleCalendar;
|
||||
@@ -12,10 +15,17 @@ internal sealed record StoredToken(string RefreshToken, string? Scope = null, st
|
||||
/// <summary>
|
||||
/// One chat's <c>token.json</c>. Reads never throw (unreadable = not connected); writes are atomic
|
||||
/// (<c>token.json.tmp</c> then a move) and owner-only on Unix.
|
||||
/// <para>
|
||||
/// With a <see cref="TokenCipher"/> the file is <c>{ "version": 2, "sealed": "..." }</c>: the agent's processes run as
|
||||
/// the bot's OS user (and owner-only modes mean nothing on some file systems), so they can read the file, but not the
|
||||
/// token. A plain file (written before encryption, or VolcanicSecretary's <c>{ "refresh_token": ... }</c>) is read once
|
||||
/// and rewritten sealed.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal sealed class GoogleTokenStore(string path)
|
||||
internal sealed class GoogleTokenStore(string path, TokenCipher? cipher = null)
|
||||
{
|
||||
public const string FileName = "token.json";
|
||||
private const int SealedVersion = 2;
|
||||
private const UnixFileMode PrivateFile = UnixFileMode.UserRead | UnixFileMode.UserWrite;
|
||||
private const UnixFileMode PrivateDirectory = UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute;
|
||||
|
||||
@@ -28,10 +38,20 @@ internal sealed class GoogleTokenStore(string path)
|
||||
if (!File.Exists(Path)) return null;
|
||||
var text = File.ReadAllText(Path);
|
||||
if (string.IsNullOrWhiteSpace(text)) return null;
|
||||
var token = VolcanicJson.Deserialize<StoredToken>(text);
|
||||
return string.IsNullOrWhiteSpace(token?.RefreshToken) ? null : token;
|
||||
if (JsonNode.Parse(text) is not JsonObject file) return null;
|
||||
if (file["sealed"] is JsonValue sealedValue)
|
||||
{
|
||||
var plain = sealedValue.TryGetValue<string>(out var data) ? cipher?.Open(data) : null;
|
||||
return plain is null ? null : Valid(VolcanicJson.Deserialize<StoredToken>(Encoding.UTF8.GetString(plain)));
|
||||
}
|
||||
|
||||
var token = Valid(VolcanicJson.Deserialize<StoredToken>(text));
|
||||
if (token is null && file["refresh_token"] is JsonValue legacy && legacy.TryGetValue<string>(out var refresh)) // VolcanicSecretary
|
||||
token = Valid(new StoredToken(refresh, GoogleOAuthClient.CalendarScope));
|
||||
if (token is not null && cipher is not null) Reseal(token);
|
||||
return token;
|
||||
}
|
||||
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException or NotSupportedException)
|
||||
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or JsonException or NotSupportedException or InvalidOperationException)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
@@ -47,7 +67,15 @@ internal sealed class GoogleTokenStore(string path)
|
||||
var options = new FileStreamOptions { Mode = FileMode.Create, Access = FileAccess.Write, Share = FileShare.None };
|
||||
if (!OperatingSystem.IsWindows()) options.UnixCreateMode = PrivateFile;
|
||||
using (var stream = new FileStream(temp, options))
|
||||
JsonSerializer.Serialize(stream, token, VolcanicJson.Indented);
|
||||
{
|
||||
if (cipher is null) JsonSerializer.Serialize(stream, token, VolcanicJson.Indented);
|
||||
else
|
||||
{
|
||||
var plain = JsonSerializer.SerializeToUtf8Bytes(token, VolcanicJson.Options);
|
||||
JsonSerializer.Serialize(stream, new SealedFile(SealedVersion, cipher.Seal(plain)), VolcanicJson.Indented);
|
||||
CryptographicOperations.ZeroMemory(plain);
|
||||
}
|
||||
}
|
||||
if (!OperatingSystem.IsWindows()) File.SetUnixFileMode(temp, PrivateFile); // a leftover .tmp keeps its old mode otherwise
|
||||
File.Move(temp, Path, overwrite: true);
|
||||
}
|
||||
@@ -57,6 +85,20 @@ internal sealed class GoogleTokenStore(string path)
|
||||
if (File.Exists(Path)) File.Delete(Path);
|
||||
}
|
||||
|
||||
private static StoredToken? Valid(StoredToken? token) => string.IsNullOrWhiteSpace(token?.RefreshToken) ? null : token;
|
||||
|
||||
/// <summary>Best effort: a plain file that can't be rewritten still works, and is sealed on a later read.</summary>
|
||||
private void Reseal(StoredToken token)
|
||||
{
|
||||
try
|
||||
{
|
||||
Write(token);
|
||||
}
|
||||
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
|
||||
{
|
||||
}
|
||||
}
|
||||
|
||||
private static void CreatePrivateDirectory(string directory)
|
||||
{
|
||||
if (Directory.Exists(directory)) return;
|
||||
@@ -67,4 +109,57 @@ internal sealed class GoogleTokenStore(string path)
|
||||
}
|
||||
Directory.CreateDirectory(directory, PrivateDirectory);
|
||||
}
|
||||
|
||||
private sealed record SealedFile(int Version, string Sealed);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// AES-256-GCM for <c>token.json</c>. The key is derived (HKDF-SHA256) from the OAuth client secret, which the agent's
|
||||
/// processes never see; the chat key is the associated data, so a file copied into another chat's folder doesn't open.
|
||||
/// A new client secret makes every stored token unreadable: each chat runs <c>/calendar_connect</c> again.
|
||||
/// Sealed form: base64 of nonce (12 bytes), tag (16 bytes), ciphertext.
|
||||
/// </summary>
|
||||
internal sealed class TokenCipher
|
||||
{
|
||||
private const int NonceSize = 12;
|
||||
private const int TagSize = 16;
|
||||
private static readonly byte[] KeyInfo = Encoding.UTF8.GetBytes("volcanic google-calendar token.json v2");
|
||||
private readonly byte[] _key;
|
||||
private readonly byte[] _chat;
|
||||
|
||||
public TokenCipher(string clientSecret, string chatKey)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrEmpty(clientSecret);
|
||||
ArgumentException.ThrowIfNullOrEmpty(chatKey);
|
||||
_key = HKDF.DeriveKey(HashAlgorithmName.SHA256, Encoding.UTF8.GetBytes(clientSecret), 32, info: KeyInfo);
|
||||
_chat = Encoding.UTF8.GetBytes(chatKey);
|
||||
}
|
||||
|
||||
public string Seal(byte[] plain)
|
||||
{
|
||||
var output = new byte[NonceSize + TagSize + plain.Length];
|
||||
var nonce = output.AsSpan(0, NonceSize);
|
||||
RandomNumberGenerator.Fill(nonce);
|
||||
using var aes = new AesGcm(_key, TagSize);
|
||||
aes.Encrypt(nonce, plain, output.AsSpan(NonceSize + TagSize), output.AsSpan(NonceSize, TagSize), _chat);
|
||||
return Convert.ToBase64String(output);
|
||||
}
|
||||
|
||||
/// <summary>Null when the data is damaged, sealed with another client secret, or sealed for another chat.</summary>
|
||||
public byte[]? Open(string data)
|
||||
{
|
||||
try
|
||||
{
|
||||
var input = Convert.FromBase64String(data);
|
||||
if (input.Length < NonceSize + TagSize) return null;
|
||||
var plain = new byte[input.Length - NonceSize - TagSize];
|
||||
using var aes = new AesGcm(_key, TagSize);
|
||||
aes.Decrypt(input.AsSpan(0, NonceSize), input.AsSpan(NonceSize + TagSize), input.AsSpan(NonceSize, TagSize), plain, _chat);
|
||||
return plain;
|
||||
}
|
||||
catch (Exception ex) when (ex is FormatException or CryptographicException)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,6 +22,7 @@ public sealed class JobScheduler : IAsyncDisposable
|
||||
private readonly SchedulerOptions _options;
|
||||
private readonly IScriptRunner _runner;
|
||||
private readonly ISecretRedactor _redactor;
|
||||
private readonly IScriptEnvironment? _scriptEnvironment;
|
||||
private readonly TimeProvider _time;
|
||||
private readonly ILogger _logger;
|
||||
private readonly Dictionary<string, JobStore> _stores = new(StringComparer.Ordinal);
|
||||
@@ -37,12 +38,14 @@ public sealed class JobScheduler : IAsyncDisposable
|
||||
private Task _pendingTick = Task.CompletedTask;
|
||||
private int _firesStarted;
|
||||
|
||||
public JobScheduler(SchedulerOptions options, IScriptRunner runner, ISecretRedactor redactor, TimeProvider time, ILogger<JobScheduler> logger)
|
||||
public JobScheduler(SchedulerOptions options, IScriptRunner runner, ISecretRedactor redactor, TimeProvider time, ILogger<JobScheduler> logger,
|
||||
IScriptEnvironment? scriptEnvironment = null)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
_options = options;
|
||||
_runner = runner;
|
||||
_redactor = redactor;
|
||||
_scriptEnvironment = scriptEnvironment;
|
||||
_time = time;
|
||||
_logger = logger;
|
||||
}
|
||||
@@ -448,16 +451,19 @@ public sealed class JobScheduler : IAsyncDisposable
|
||||
private async Task<JobResult?> RunWatcherAsync(ScheduledJob job, IJobDelivery delivery, TimeZoneInfo zone, CancellationToken scriptToken)
|
||||
{
|
||||
var timeout = job.TimeoutSeconds is { } seconds ? TimeSpan.FromSeconds(seconds) : _options.DefaultScriptTimeout;
|
||||
var request = new ScriptRequest(job.Command!, delivery.WorkspaceFor(job.Target), timeout, job.Target, job.Id, ZoneNames.Iana(zone));
|
||||
IReadOnlyDictionary<string, string>? extra;
|
||||
ScriptOutcome outcome;
|
||||
try
|
||||
{
|
||||
extra = await ScriptEnvironmentAsync(job, timeout, scriptToken).ConfigureAwait(false);
|
||||
var request = new ScriptRequest(job.Command!, delivery.WorkspaceFor(job.Target), timeout, job.Target, job.Id, ZoneNames.Iana(zone), extra);
|
||||
outcome = await _runner.RunAsync(request, scriptToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (OperationCanceledException) when (scriptToken.IsCancellationRequested)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
string Redact(string text) => RedactRun(text, extra);
|
||||
if (scriptToken.IsCancellationRequested || !Exists(job.Id)) return null;
|
||||
|
||||
var at = _time.GetUtcNow();
|
||||
@@ -465,11 +471,11 @@ public sealed class JobScheduler : IAsyncDisposable
|
||||
{
|
||||
case ScriptOutcome.Output output:
|
||||
// The condition is met: exactly one agent prompt. Redacted: watcher stdout isn't a tool result.
|
||||
return await DeliverAsync(job, ct => delivery.RunPromptAsync(job.Target, JobMessages.WatcherMatch(job, _redactor.Redact(output.Text)), ct)).ConfigureAwait(false)
|
||||
return await DeliverAsync(job, ct => delivery.RunPromptAsync(job.Target, JobMessages.WatcherMatch(job, Redact(output.Text)), ct)).ConfigureAwait(false)
|
||||
? new JobResult(true, 0, false, null, at)
|
||||
: null;
|
||||
case ScriptOutcome.Failed failed:
|
||||
var result = new JobResult(false, failed.ExitCode, failed.TimedOut, _redactor.Redact(failed.StderrTail), at);
|
||||
var result = new JobResult(false, failed.ExitCode, failed.TimedOut, Redact(failed.StderrTail), at);
|
||||
if (!IsRepeat(LastResult(job.Id), result)
|
||||
&& !await DeliverAsync(job, ct => delivery.SendTextAsync(job.Target, JobMessages.Failure(job, result), ct)).ConfigureAwait(false))
|
||||
return null; // the alert was cut off by shutdown: not recorded, so the next failure alerts again
|
||||
@@ -479,6 +485,32 @@ public sealed class JobScheduler : IAsyncDisposable
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>The host's extra variables for this run; null when there are none or the host failed (logged).</summary>
|
||||
private async Task<IReadOnlyDictionary<string, string>?> ScriptEnvironmentAsync(ScheduledJob job, TimeSpan timeout, CancellationToken scriptToken)
|
||||
{
|
||||
if (_scriptEnvironment is null) return null;
|
||||
try
|
||||
{
|
||||
var extra = await _scriptEnvironment.ForRunAsync(job.Target, timeout, scriptToken).ConfigureAwait(false);
|
||||
return extra.Count == 0 ? null : extra;
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException || !scriptToken.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogWarning(ex, "Job {Id} ({Name}): the script environment failed; the watcher runs without it", job.Id, job.Name);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>The host's secrets, then this run's extra values (8 characters or more, like the policy's secret values).</summary>
|
||||
private string RedactRun(string text, IReadOnlyDictionary<string, string>? extra)
|
||||
{
|
||||
text = _redactor.Redact(text);
|
||||
if (extra is null) return text;
|
||||
foreach (var value in extra.Values.Where(v => v.Length >= 8).OrderByDescending(v => v.Length))
|
||||
text = text.Replace(value, "[REDACTED]", StringComparison.Ordinal);
|
||||
return text;
|
||||
}
|
||||
|
||||
/// <summary>The previous run failed the same way (exit code and timeout): the alert isn't sent again.</summary>
|
||||
private static bool IsRepeat(JobResult? previous, JobResult current) =>
|
||||
previous is { Ok: false } p && p.ExitCode == current.ExitCode && p.TimedOut == current.TimedOut;
|
||||
|
||||
@@ -7,8 +7,22 @@ using Volcanic.AgenticCore.Abstractions.Processes;
|
||||
|
||||
namespace Volcanic.AgenticCore.Extensions.Scheduler;
|
||||
|
||||
/// <summary>One watcher run. <see cref="TimeZoneId"/> becomes the script's <c>TZ</c> (the target's zone).</summary>
|
||||
public sealed record ScriptRequest(string Command, string Cwd, TimeSpan Timeout, string Target, string JobId, string? TimeZoneId);
|
||||
/// <summary>
|
||||
/// One watcher run. <see cref="TimeZoneId"/> becomes the script's <c>TZ</c> (the target's zone); <see cref="Environment"/>
|
||||
/// (from the host's <see cref="IScriptEnvironment"/>) is set after the process environment filter.
|
||||
/// </summary>
|
||||
public sealed record ScriptRequest(string Command, string Cwd, TimeSpan Timeout, string Target, string JobId, string? TimeZoneId,
|
||||
IReadOnlyDictionary<string, string>? Environment = null);
|
||||
|
||||
/// <summary>
|
||||
/// Extra variables for one watcher run, e.g. a short-lived access token. They are set after the process environment
|
||||
/// filter (so a deny rule doesn't remove them), and their values are redacted from the run's output and failure alerts.
|
||||
/// </summary>
|
||||
public interface IScriptEnvironment
|
||||
{
|
||||
/// <summary>Variables for a run of <paramref name="target"/>'s watcher that may last <paramref name="timeout"/>; empty adds nothing. A throw is logged and the run goes ahead without them.</summary>
|
||||
Task<IReadOnlyDictionary<string, string>> ForRunAsync(string target, TimeSpan timeout, CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>What a watcher run means for the job.</summary>
|
||||
public abstract record ScriptOutcome
|
||||
@@ -60,6 +74,7 @@ public sealed class ScriptRunner(SchedulerOptions options, IProcessSpawnPolicy p
|
||||
Scope = request.Target,
|
||||
Origin = request.JobId,
|
||||
TimeZoneId = request.TimeZoneId,
|
||||
Environment = request.Environment?.ToDictionary(e => e.Key, e => (string?)e.Value),
|
||||
});
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -95,6 +95,8 @@ builder.Services.AddSingleton<IUpdateRouter, UpdateRouter>();
|
||||
builder.Services.AddSingleton<PollingService>();
|
||||
builder.Services.AddHostedService(sp => sp.GetRequiredService<PollingService>());
|
||||
if (scheduling.Enabled) SchedulerSetup.Register(builder.Services); // the hosted service starts after PollingService
|
||||
// Watchers get a short-lived GOOGLE_ACCESS_TOKEN for their chat's calendar, never the client secret or refresh token.
|
||||
if (scheduling.Enabled && calendar is not null) builder.Services.AddSingleton<IScriptEnvironment, CalendarScriptEnvironment>();
|
||||
builder.Services.AddHostedService<BotSetupService>();
|
||||
|
||||
using var host = builder.Build();
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
using System.Globalization;
|
||||
using Volcanic.AgenticCore.Extensions.GoogleCalendar;
|
||||
using Volcanic.AgenticCore.Extensions.Scheduler;
|
||||
using Volcanic.AgenticCore.Telegram.Chats;
|
||||
|
||||
namespace Volcanic.AgenticCore.Telegram.Scheduling;
|
||||
|
||||
/// <summary>
|
||||
/// Gives a chat's watcher scripts <c>GOOGLE_ACCESS_TOKEN</c>: an access token for that chat's calendar, valid for the
|
||||
/// run's timeout plus a minute (as far as Google's one-hour lifetime allows). The bot refreshes it; the refresh token
|
||||
/// and the client secret never reach the script. A chat without a connected calendar gets nothing.
|
||||
/// </summary>
|
||||
public sealed class CalendarScriptEnvironment(ChatRegistry chats, GoogleCalendarOptions calendar) : IScriptEnvironment
|
||||
{
|
||||
public const string AccessTokenVariable = "GOOGLE_ACCESS_TOKEN";
|
||||
private static readonly TimeSpan Margin = TimeSpan.FromMinutes(1);
|
||||
private static readonly IReadOnlyDictionary<string, string> None = new Dictionary<string, string>();
|
||||
|
||||
public async Task<IReadOnlyDictionary<string, string>> ForRunAsync(string target, TimeSpan timeout, CancellationToken cancellationToken)
|
||||
{
|
||||
if (!long.TryParse(target, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture, out var chatId)) return None;
|
||||
var token = await (calendar.Connections ?? GoogleCalendarConnections.Shared)
|
||||
.TryGetAccessTokenAsync(calendar, ChatRegistry.ScopeFor(chatId), chats.WorkspaceFor(chatId), timeout + Margin, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
return token is null ? None : new Dictionary<string, string> { [AccessTokenVariable] = token };
|
||||
}
|
||||
}
|
||||
@@ -129,6 +129,56 @@ public class GoogleCalendarConnectionTests : IDisposable
|
||||
Assert.Equal(2, Refreshes);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_longer_validity_refreshes_a_token_that_would_expire_during_the_run()
|
||||
{
|
||||
var connection = Connected();
|
||||
await connection.GetAccessTokenAsync(CancellationToken.None);
|
||||
_http.Json("POST /token", """{"access_token":"at-2","expires_in":3600}""");
|
||||
_time.Advance(TimeSpan.FromMinutes(50)); // 10 minutes left
|
||||
|
||||
Assert.Equal("at-1", await connection.GetAccessTokenAsync(CancellationToken.None)); // enough for the tool
|
||||
Assert.Equal("at-2", await connection.GetAccessTokenAsync(CancellationToken.None, TimeSpan.FromMinutes(15))); // not for a 15-minute run
|
||||
Assert.Equal(2, Refreshes);
|
||||
}
|
||||
|
||||
private GoogleCalendarOptions Options(string? secret = "csecret") =>
|
||||
new() { ClientId = "cid", ClientSecret = secret, TokenDirectory = _root, Http = _http.CreateClient(), Time = _time, GetEnv = _ => null };
|
||||
|
||||
[Fact]
|
||||
public async Task TryGetAccessTokenAsync_returns_the_chats_token_or_null_when_not_connected()
|
||||
{
|
||||
Assert.Null(await _connections.TryGetAccessTokenAsync(Options(), "telegram-1", _root, TimeSpan.FromMinutes(6), CancellationToken.None));
|
||||
Connected();
|
||||
|
||||
Assert.Equal("at-1", await _connections.TryGetAccessTokenAsync(Options(), "telegram-1", _root, TimeSpan.FromMinutes(6), CancellationToken.None));
|
||||
Assert.Null(await _connections.TryGetAccessTokenAsync(Options(secret: null), "telegram-1", _root, TimeSpan.FromMinutes(6), CancellationToken.None));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TryGetAccessTokenAsync_returns_null_when_Google_rejects_the_token()
|
||||
{
|
||||
Connected();
|
||||
_http.Json("POST /token", (HttpStatusCode.BadRequest, """{"error":"invalid_grant"}"""));
|
||||
|
||||
Assert.Null(await _connections.TryGetAccessTokenAsync(Options(), "telegram-1", _root, TimeSpan.FromMinutes(6), CancellationToken.None));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void The_token_file_is_sealed_for_its_chat()
|
||||
{
|
||||
var connection = Connected(refreshToken: "1//0e-secret-refresh");
|
||||
|
||||
var json = File.ReadAllText(connection.Store.Path);
|
||||
Assert.DoesNotContain("1//0e-secret-refresh", json);
|
||||
Assert.Contains("\"sealed\"", json);
|
||||
Assert.Equal("1//0e-secret-refresh", connection.Store.Read()!.RefreshToken);
|
||||
var other = _connections.For(_settings, "telegram-2", _root);
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(other.Store.Path)!);
|
||||
File.Copy(connection.Store.Path, other.Store.Path); // another chat's agent copies the file into its folder
|
||||
Assert.Null(other.Store.Read());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Invalid_grant_throws_not_connected_and_keeps_the_file()
|
||||
{
|
||||
|
||||
@@ -78,6 +78,60 @@ public class GoogleTokenStoreTests : IDisposable
|
||||
Assert.Null(store.Read());
|
||||
}
|
||||
|
||||
private static TokenCipher Cipher(string secret = "client-secret", string chat = "telegram-1") => new(secret, chat);
|
||||
|
||||
[Fact]
|
||||
public void A_sealed_file_round_trips_and_never_shows_the_token()
|
||||
{
|
||||
var store = new GoogleTokenStore(TokenPath, Cipher());
|
||||
store.Write(Sample);
|
||||
|
||||
Assert.Equal(Sample, store.Read());
|
||||
var json = File.ReadAllText(TokenPath);
|
||||
Assert.DoesNotContain("1//0e-refresh", json);
|
||||
Assert.DoesNotContain("user@gmail.com", json);
|
||||
Assert.Contains("\"version\": 2", json);
|
||||
Assert.Null(new GoogleTokenStore(TokenPath).Read()); // no key, no token
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_sealed_file_does_not_open_with_another_client_secret_or_for_another_chat()
|
||||
{
|
||||
new GoogleTokenStore(TokenPath, Cipher()).Write(Sample);
|
||||
|
||||
Assert.Null(new GoogleTokenStore(TokenPath, Cipher(secret: "rotated-secret")).Read());
|
||||
Assert.Null(new GoogleTokenStore(TokenPath, Cipher(chat: "telegram-2")).Read());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_tampered_sealed_file_reads_as_not_connected()
|
||||
{
|
||||
new GoogleTokenStore(TokenPath, Cipher()).Write(Sample);
|
||||
var json = File.ReadAllText(TokenPath);
|
||||
var data = System.Text.Json.Nodes.JsonNode.Parse(json)!["sealed"]!.GetValue<string>();
|
||||
var bytes = Convert.FromBase64String(data);
|
||||
bytes[^1] ^= 1;
|
||||
File.WriteAllText(TokenPath, json.Replace(data, Convert.ToBase64String(bytes)));
|
||||
|
||||
Assert.Null(new GoogleTokenStore(TokenPath, Cipher()).Read());
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("""{"refreshToken":"1//0e-plain","account":"user@gmail.com"}""", "user@gmail.com")]
|
||||
[InlineData("""{"refresh_token":"1//0e-plain"}""", null)] // VolcanicSecretary
|
||||
public void A_plain_file_is_read_and_rewritten_sealed(string content, string? account)
|
||||
{
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(TokenPath)!);
|
||||
File.WriteAllText(TokenPath, content);
|
||||
|
||||
var token = new GoogleTokenStore(TokenPath, Cipher()).Read();
|
||||
|
||||
Assert.Equal("1//0e-plain", token!.RefreshToken);
|
||||
Assert.Equal(account, token.Account);
|
||||
Assert.DoesNotContain("1//0e-plain", File.ReadAllText(TokenPath));
|
||||
Assert.Equal(token, new GoogleTokenStore(TokenPath, Cipher()).Read());
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ToString_never_shows_the_refresh_token()
|
||||
{
|
||||
|
||||
@@ -462,4 +462,37 @@ public sealed class JobSchedulerTests
|
||||
var stored = (await h.Store.LoadAsync(CancellationToken.None)).Concat(await h.StoreFor("222").LoadAsync(CancellationToken.None)).Select(j => j.Id);
|
||||
Assert.Equal(ids.Order(), stored.Order()); // the new id is saved
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_watcher_gets_the_hosts_extra_variables_and_their_values_are_redacted()
|
||||
{
|
||||
var environment = new FakeScriptEnvironment();
|
||||
environment.Variables["GOOGLE_ACCESS_TOKEN"] = "ya29.short-lived";
|
||||
await using var h = new SchedulerHarness(environment: environment);
|
||||
await h.StartAsync();
|
||||
var id = (await h.CreateAsync(Watcher())).Job.Id;
|
||||
h.Runner.Then(new ScriptOutcome.Output("token ya29.short-lived"), new ScriptOutcome.Failed(1, "401 for ya29.short-lived", false));
|
||||
|
||||
await h.AdvanceAsync(Minute);
|
||||
await h.AdvanceAsync(Minute);
|
||||
|
||||
Assert.Equal(("111", TimeSpan.FromSeconds(300)), environment.Calls[0]);
|
||||
Assert.Equal("ya29.short-lived", h.Runner.Requests[0].Environment!["GOOGLE_ACCESS_TOKEN"]);
|
||||
Assert.Equal($"[Scheduled job \"Price check\" ({id})] The watcher script printed:\ntoken [REDACTED]", Assert.Single(h.Delivery.Prompts).Prompt);
|
||||
Assert.Contains("401 for [REDACTED]", Assert.Single(h.Delivery.Texts).Text);
|
||||
Assert.Equal("401 for [REDACTED]", Assert.Single(await h.Store.LoadAsync(CancellationToken.None)).LastResult!.StderrTail);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_failing_script_environment_does_not_stop_the_run()
|
||||
{
|
||||
var environment = new FakeScriptEnvironment { Fail = new HttpRequestException("Google is down") };
|
||||
await using var h = new SchedulerHarness(environment: environment);
|
||||
await h.StartAsync();
|
||||
await h.CreateAsync(Watcher());
|
||||
|
||||
await h.AdvanceAsync(Minute);
|
||||
|
||||
Assert.Null(Assert.Single(h.Runner.Requests).Environment);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -123,6 +123,18 @@ public sealed class ScriptRunnerTests : IDisposable
|
||||
Assert.Equal(new ScriptOutcome.Output("unset|unset|Asia/Tokyo"), outcome);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Request_variables_reach_the_script_past_the_deny_rules()
|
||||
{
|
||||
if (OperatingSystem.IsWindows()) return;
|
||||
var request = Request("printf '%s|%s' \"$GOOGLE_ACCESS_TOKEN\" \"${OPENAI_API_KEY-unset}\"") with
|
||||
{
|
||||
Environment = new Dictionary<string, string> { ["GOOGLE_ACCESS_TOKEN"] = "ya29.short-lived" }, // matches *_TOKEN
|
||||
};
|
||||
|
||||
Assert.Equal(new ScriptOutcome.Output("ya29.short-lived|unset"), await Runner().RunAsync(request, CancellationToken.None));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_throwing_host_hook_fails_without_starting_a_process()
|
||||
{
|
||||
|
||||
@@ -115,6 +115,22 @@ internal sealed class FakeScriptRunner : IScriptRunner
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Returns <see cref="Variables"/> (or throws <see cref="Fail"/>) and records each call.</summary>
|
||||
internal sealed class FakeScriptEnvironment : IScriptEnvironment
|
||||
{
|
||||
private readonly List<(string Target, TimeSpan Timeout)> _calls = [];
|
||||
public Dictionary<string, string> Variables { get; } = new();
|
||||
public Exception? Fail { get; set; }
|
||||
public IReadOnlyList<(string Target, TimeSpan Timeout)> Calls { get { lock (_calls) return _calls.ToList(); } }
|
||||
|
||||
public Task<IReadOnlyDictionary<string, string>> ForRunAsync(string target, TimeSpan timeout, CancellationToken cancellationToken)
|
||||
{
|
||||
lock (_calls) _calls.Add((target, timeout));
|
||||
if (Fail is { } error) throw error;
|
||||
return Task.FromResult<IReadOnlyDictionary<string, string>>(new Dictionary<string, string>(Variables));
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class FakeRedactor(params string[] secrets) : ISecretRedactor
|
||||
{
|
||||
public string Redact(string text) => secrets.Aggregate(text, (t, s) => t.Replace(s, "[REDACTED]", StringComparison.Ordinal));
|
||||
@@ -127,13 +143,13 @@ internal sealed class SchedulerHarness : IAsyncDisposable
|
||||
public static readonly TimeZoneInfo Berlin = TimeZoneInfo.FindSystemTimeZoneById("Europe/Berlin");
|
||||
public static readonly TimeZoneInfo Tokyo = TimeZoneInfo.FindSystemTimeZoneById("Asia/Tokyo");
|
||||
|
||||
public SchedulerHarness(DateTimeOffset? start = null, Action<SchedulerOptions>? configure = null)
|
||||
public SchedulerHarness(DateTimeOffset? start = null, Action<SchedulerOptions>? configure = null, IScriptEnvironment? environment = null)
|
||||
{
|
||||
Time = new FakeTimeProvider(start ?? Start);
|
||||
Options = new SchedulerOptions { LegacyStorePath = Path.Combine(Root, "scheduler", "jobs.json") };
|
||||
configure?.Invoke(Options);
|
||||
Delivery.WorkspaceRoot = Path.Combine(Root, "workspaces");
|
||||
Scheduler = new JobScheduler(Options, Runner, Redactor, Time, NullLogger<JobScheduler>.Instance);
|
||||
Scheduler = new JobScheduler(Options, Runner, Redactor, Time, NullLogger<JobScheduler>.Instance, environment);
|
||||
}
|
||||
|
||||
public string Root { get; } = Directory.CreateTempSubdirectory("scheduler-").FullName;
|
||||
|
||||
@@ -160,4 +160,26 @@ public class GoogleCalendarHostTests : IDisposable
|
||||
}, "the connected message");
|
||||
Assert.True(File.Exists(Path.Combine(_tokens, ChatRegistry.ScopeFor(Updates.AllowedUser), "token.json")));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Watchers_get_a_short_lived_access_token_and_the_token_file_stays_sealed()
|
||||
{
|
||||
var calendar = Calendar();
|
||||
await using var host = new ChatHost(calendar: calendar);
|
||||
var environment = new Volcanic.AgenticCore.Telegram.Scheduling.CalendarScriptEnvironment(host.Chats, calendar);
|
||||
var target = Updates.AllowedUser.ToString(System.Globalization.CultureInfo.InvariantCulture);
|
||||
Assert.Empty(await environment.ForRunAsync(target, TimeSpan.FromMinutes(5), CancellationToken.None)); // not connected yet
|
||||
|
||||
await host.SendAsync("/calendar_connect");
|
||||
await Wait.UntilAsync(() =>
|
||||
{
|
||||
_time.Advance(TimeSpan.FromSeconds(5));
|
||||
return host.Texts.Any(t => t.Contains("Google Calendar connected as owner@example.com"));
|
||||
}, "the connected message");
|
||||
|
||||
var variables = await environment.ForRunAsync(target, TimeSpan.FromMinutes(5), CancellationToken.None);
|
||||
Assert.Equal("at-1", Assert.Single(variables, v => v.Key == "GOOGLE_ACCESS_TOKEN").Value);
|
||||
Assert.DoesNotContain("rt-1", File.ReadAllText(Path.Combine(_tokens, ChatRegistry.ScopeFor(Updates.AllowedUser), "token.json")));
|
||||
Assert.Empty(await environment.ForRunAsync("not-a-chat", TimeSpan.FromMinutes(5), CancellationToken.None));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user