ayrisdev/screenclipper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: analiz zincirini atlayıp ham segmenti doğrudan Telegram'a gönderme modu

Browse Source 
Yeni "Ham Kayıt Modu" ayarı (send_raw_segments_to_telegram, varsayılan
kapalı): açıksa her ham segment tamamlanır tamamlanmaz sinyal analizi/
STT/9:16 render zincirine hiç girmeden doğrudan Telegram'a gönderiliyor
ve sunucudan siliniyor. Kullanıcı transkript/9:16 istemediğinde ("sen
bana direk videoyu gönder") tüm analiz pipeline'ını atlayıp sadece
kayıt+gönder+sil yapan basit bir mod.

- RawSegmentStatus'a SENT_TO_TELEGRAM eklendi, RawSegment.filePath
  nullable yapıldı (aynı ShortVideo.filePath deseni — gönderim sonrası
  null, dosya diskten silinmiş demek).
- api-daemon/src/telegram.ts'e sendTelegramVideo eklendi (native fetch+
  FormData+Blob ile multipart upload, ~50MB bot-limiti önceden kontrol
  ediliyor) — worker/telegram.py'deki aynı fonksiyonun Node karşılığı.
- streamIngest.ts: segment tamamlanınca, mod açıksa signal-detection
  kuyruğuna hiç eklemeden doğrudan gönderiyor (kuyruğa eklemek, aşağıda
  dosyayı silmenin sinyal analizini bozmasına sebep olurdu).
- Panel: segment listelerinde filePath=null durumunda video/indir yerine
  "Telegram'a gönderildi" notu (segments, channels/[id]).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
ayrisdev
2026-09-04 19:57:14 +03:00
co-authored by Claude Sonnet 5
parent 9aed4e891e
commit 11e98b300e
10 changed files with 193 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/api-daemon/src/capture/streamIngest.ts
+35 -9
View File
@@ -1,5 +1,5 @@
import { spawn, type ChildProcess } from "node:child_process";
import { mkdir, readFile } from "node:fs/promises";
import { mkdir, readFile, unlink } from "node:fs/promises";
import path from "node:path";
import { Worker, type Job } from "bullmq";
import { prisma } from "@streamclipper/db";
@@ -7,6 +7,7 @@ import { env } from "../env";
import { QUEUE_NAMES, signalDetectionQueue, type StreamIngestJob } from "../queues";
import { redisConnection } from "../redis";
import { ytdlpAntiBotArgs } from "../ytdlpCookies";
import { sendTelegramVideo } from "../telegram";
const SEGMENT_LIST_POLL_MS = 5_000;
@@ -59,13 +60,20 @@ function parseSegmentListLines(raw: string): SegmentListEntry[] {
});
}
async function sendRawSegmentsToTelegramEnabled(): Promise<boolean> {
const setting = await prisma.appSetting.findUnique({ where: { key: "send_raw_segments_to_telegram" } });
return setting?.value === "true";
}
async function processCompletedSegments(
entries: SegmentListEntry[],
fromIndex: number,
outDir: string,
sessionId: string,
channelName: string,
): Promise<number> {
let processed = fromIndex;
const sendRawToTelegram = await sendRawSegmentsToTelegramEnabled();
for (let i = fromIndex; i < entries.length; i++) {
const entry = entries[i];
@@ -82,13 +90,29 @@ async function processCompletedSegments(
},
});
await signalDetectionQueue.add("analyze-segment", {
rawSegmentId: segment.id,
filePath,
sessionId,
});
if (sendRawToTelegram) {
// Kullanıcı transkript/9:16 zincirini hiç istemiyor — ham segmenti
// analiz kuyruğuna hiç sokmadan doğrudan gönderiyoruz. Kuyruğa
// eklemiyoruz çünkü aşağıda dosyayı silmek, o segmenti daha sonra
// okumaya çalışacak sinyal analizini bozardı.
const caption = `🎬 *${channelName}* — ${duration}sn ham kayıt`;
const sent = await sendTelegramVideo(filePath, caption);
if (sent) {
await unlink(filePath).catch(() => {});
await prisma.rawSegment.update({ where: { id: segment.id }, data: { status: "SENT_TO_TELEGRAM", filePath: null } });
console.log(`[stream-ingest] segment ${entry.fileName} sent to Telegram and removed from disk`);
} else {
console.warn(`[stream-ingest] segment ${entry.fileName} failed to send to Telegram, kept on disk`);
}
} else {
await signalDetectionQueue.add("analyze-segment", {
rawSegmentId: segment.id,
filePath,
sessionId,
});
console.log(`[stream-ingest] segment ready: ${entry.fileName} -> raw_segment ${segment.id}`);
}
console.log(`[stream-ingest] segment ready: ${entry.fileName} -> raw_segment ${segment.id}`);
processed = i + 1;
}
@@ -97,6 +121,8 @@ async function processCompletedSegments(
async function runCapture(job: Job<StreamIngestJob>): Promise<void> {
const { sessionId, youtubeUrl, segmentTimeSec, channelDbId } = job.data;
const channel = await prisma.channel.findUnique({ where: { id: channelDbId } });
const channelName = channel?.name ?? "Bilinmeyen kanal";
const outDir = path.join(env.sharedMediaRoot, "raw", sessionId);
await mkdir(outDir, { recursive: true });
@@ -150,7 +176,7 @@ async function runCapture(job: Job<StreamIngestJob>): Promise<void> {
if (!raw) return;
const entries = parseSegmentListLines(raw);
if (entries.length > lastProcessedIndex) {
lastProcessedIndex = await processCompletedSegments(entries, lastProcessedIndex, outDir, sessionId);
lastProcessedIndex = await processCompletedSegments(entries, lastProcessedIndex, outDir, sessionId, channelName);
await prisma.streamSession.update({
where: { id: sessionId },
data: { totalSegments: lastProcessedIndex },
@@ -178,7 +204,7 @@ async function runCapture(job: Job<StreamIngestJob>): Promise<void> {
const raw = await readFile(segmentListPath, "utf8").catch(() => "");
const entries = parseSegmentListLines(raw);
if (entries.length > lastProcessedIndex) {
lastProcessedIndex = await processCompletedSegments(entries, lastProcessedIndex, outDir, sessionId);
lastProcessedIndex = await processCompletedSegments(entries, lastProcessedIndex, outDir, sessionId, channelName);
}
await prisma.streamSession.update({
apps/api-daemon/src/cleanup.ts
+1 -1
View File
@@ -32,7 +32,7 @@ async function cleanupOnce(): Promise<void> {
for (const segment of staleSegments) {
await prisma.candidateSegment.deleteMany({ where: { rawSegmentId: segment.id } });
await prisma.rawSegment.delete({ where: { id: segment.id } });
await unlink(segment.filePath).catch(() => {});
if (segment.filePath) await unlink(segment.filePath).catch(() => {});
}
if (staleSegments.length > 0) {
apps/api-daemon/src/telegram.ts
+43
View File
@@ -1,6 +1,12 @@
import { readFile, stat } from "node:fs/promises";
import path from "node:path";
const BOT_TOKEN = process.env.TELEGRAM_BOT_TOKEN ?? "";
const CHAT_ID = process.env.TELEGRAM_CHAT_ID ?? "";
// Telegram's bot-upload limit (no local Bot API server).
const MAX_VIDEO_BYTES = 50 * 1024 * 1024;
export async function sendTelegramMessage(text: string): Promise<void> {
if (!BOT_TOKEN || !CHAT_ID) {
console.warn("[telegram] TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID not set, skipping notification:", text);
@@ -18,3 +24,40 @@ export async function sendTelegramMessage(text: string): Promise<void> {
console.error("[telegram] failed to send message:", res.status, await res.text());
}
}
/** Uploads a video file to Telegram. Returns false (never throws) on any failure — the caller decides what that means for the local file. */
export async function sendTelegramVideo(filePath: string, caption: string): Promise<boolean> {
if (!BOT_TOKEN || !CHAT_ID) {
console.warn("[telegram] TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID not set, skipping video:", caption);
return false;
}
const { size } = await stat(filePath);
if (size > MAX_VIDEO_BYTES) {
console.warn(`[telegram] video ${filePath} is ${size} bytes, over Telegram's bot-upload limit — skipping`);
return false;
}
try {
const buffer = await readFile(filePath);
const form = new FormData();
form.set("chat_id", CHAT_ID);
form.set("caption", caption);
form.set("parse_mode", "Markdown");
form.set("video", new Blob([buffer], { type: "video/mp4" }), path.basename(filePath));
const res = await fetch(`https://api.telegram.org/bot${BOT_TOKEN}/sendVideo`, {
method: "POST",
body: form,
});
if (!res.ok) {
console.error("[telegram] failed to send video:", res.status, await res.text());
return false;
}
return true;
} catch (err) {
console.error("[telegram] failed to upload video:", err);
return false;
}
}
apps/frontend/app/actions.ts
+12 -2
View File
@@ -40,7 +40,7 @@ export async function deleteRawSegment(segmentId: string) {
await deleteShortsForCandidates(segment.candidates.map((c) => c.id));
await prisma.candidateSegment.deleteMany({ where: { rawSegmentId: segmentId } });
await prisma.rawSegment.delete({ where: { id: segmentId } });
await unlink(segment.filePath).catch(() => {});
if (segment.filePath) await unlink(segment.filePath).catch(() => {});
revalidatePath("/segments");
revalidatePath("/", "layout");
@@ -61,7 +61,7 @@ export async function deleteChannel(channelId: string) {
});
const rawSegmentIds = sessions.flatMap((s) => s.segments.map((seg) => seg.id));
const candidateIds = sessions.flatMap((s) => s.segments.flatMap((seg) => seg.candidates.map((c) => c.id)));
const filePaths = sessions.flatMap((s) => s.segments.map((seg) => seg.filePath));
const filePaths = sessions.flatMap((s) => s.segments.map((seg) => seg.filePath)).filter((p): p is string => Boolean(p));
await deleteShortsForCandidates(candidateIds);
await prisma.candidateSegment.deleteMany({ where: { rawSegmentId: { in: rawSegmentIds } } });
@@ -175,6 +175,16 @@ export async function updateDeleteAfterTelegramSend(formData: FormData) {
revalidatePath("/settings");
}
export async function updateSendRawSegmentsToTelegram(formData: FormData) {
const enabled = formData.get("sendRawSegmentsToTelegram") === "true";
await prisma.appSetting.upsert({
where: { key: "send_raw_segments_to_telegram" },
create: { key: "send_raw_segments_to_telegram", value: String(enabled) },
update: { value: String(enabled) },
});
revalidatePath("/settings");
}
export async function updateNotificationPrefs(formData: FormData) {
const streamStart = formData.get("notifyStreamStart") === "true";
const renderDone = formData.get("notifyRenderDone") === "true";
apps/frontend/app/api/segments/[id]/video/route.ts
+1 -1
View File
@@ -6,7 +6,7 @@ export async function GET(req: NextRequest, { params }: { params: Promise<{ id:
const { id } = await params;
const segment = await prisma.rawSegment.findUnique({ where: { id } });
if (!segment) return new Response("Not found", { status: 404 });
if (!segment?.filePath) return new Response("Not found", { status: 404 });
return streamVideoFile(req, segment.filePath, `${id}.mp4`);
}
apps/frontend/app/channels/[id]/page.tsx
+35 -21
View File
@@ -30,6 +30,7 @@ const STATUS_BADGE: Record<string, VariantProps<typeof badgeVariants>["variant"]
PENDING: "muted",
PROCESSED: "ok",
DISCARDED: "err",
SENT_TO_TELEGRAM: "ok",
PENDING_STT: "warn",
TRANSCRIBED: "ok",
RENDERING: "warn",
@@ -225,30 +226,43 @@ export default async function ChannelDetailPage({ params }: { params: Promise<{
</span>
<Badge variant={STATUS_BADGE[segment.status] ?? "muted"}>{segment.status}</Badge>
</div>
<span className="font-mono-num text-[0.7rem] text-muted-foreground">{segment.filePath}</span>
<video controls preload="metadata" className="mt-2 w-full max-w-md rounded-md border border-border">
<source src={`/api/segments/${segment.id}/video`} type="video/mp4" />
</video>
<div className="mt-2 flex items-center gap-2">
<Button asChild variant="outline" size="sm">
<a href={`/api/segments/${segment.id}/video?download=1`}>İndir</a>
</Button>
<form action={deleteRawSegment.bind(null, segment.id)}>
<ConfirmButton
confirmText="Bu segmenti ve içindeki tüm aday klipleri silmek istediğine emin misin? Geri alınamaz."
label="Sil"
/>
</form>
</div>
{segment.candidates.length === 0 && segment.status !== "PENDING" && (
<p className="mt-3 text-xs text-muted-foreground">
Bu segmentte aday klip bulunamadı (sinyal analizi ilginç bir an tespit etmedi).
{segment.filePath ? (
<>
<span className="font-mono-num text-[0.7rem] text-muted-foreground">{segment.filePath}</span>
<video controls preload="metadata" className="mt-2 w-full max-w-md rounded-md border border-border">
<source src={`/api/segments/${segment.id}/video`} type="video/mp4" />
</video>
<div className="mt-2 flex items-center gap-2">
<Button asChild variant="outline" size="sm">
<a href={`/api/segments/${segment.id}/video?download=1`}>İndir</a>
</Button>
<form action={deleteRawSegment.bind(null, segment.id)}>
<ConfirmButton
confirmText="Bu segmenti ve içindeki tüm aday klipleri silmek istediğine emin misin? Geri alınamaz."
label="Sil"
/>
</form>
</div>
</>
) : (
<p className="mt-1 text-xs text-muted-foreground">
✅ Telegram&apos;a gönderildi, sunucuda saklanmıyor.
</p>
)}
{segment.status === "SENT_TO_TELEGRAM" ? (
<p className="mt-3 text-xs text-muted-foreground">
Analiz edilmeden (transkript/9:16 atlanarak) doğrudan gönderildi.
</p>
) : (
segment.candidates.length === 0 &&
segment.status !== "PENDING" && (
<p className="mt-3 text-xs text-muted-foreground">
Bu segmentte aday klip bulunamadı (sinyal analizi ilginç bir an tespit etmedi).
</p>
)
)}
{segment.candidates.length > 0 && (
<div className="mt-3 flex flex-col gap-2">
{segment.candidates.map((c) => (
apps/frontend/app/segments/page.tsx
+26 -16
View File
@@ -13,6 +13,7 @@ const STATUS_BADGE: Record<string, VariantProps<typeof badgeVariants>["variant"]
PENDING: "muted",
PROCESSED: "ok",
DISCARDED: "err",
SENT_TO_TELEGRAM: "ok",
PENDING_STT: "warn",
TRANSCRIBED: "ok",
RENDERING: "warn",
@@ -53,23 +54,32 @@ export default async function SegmentsPage() {
<Badge variant={STATUS_BADGE[segment.status] ?? "muted"}>{segment.status}</Badge>
</CardHeader>
<CardContent className="flex flex-col gap-3">
<video controls preload="metadata" className="w-full max-w-md rounded-md border border-border">
<source src={`/api/segments/${segment.id}/video`} type="video/mp4" />
</video>
{segment.filePath ? (
<>
<video controls preload="metadata" className="w-full max-w-md rounded-md border border-border">
<source src={`/api/segments/${segment.id}/video`} type="video/mp4" />
</video>
<div className="flex items-center gap-2">
<Button asChild variant="outline" size="sm">
<a href={`/api/segments/${segment.id}/video?download=1`}>İndir</a>
</Button>
<form action={deleteRawSegment.bind(null, segment.id)}>
<ConfirmButton
confirmText="Bu segmenti ve içindeki tüm aday klipleri silmek istediğine emin misin? Geri alınamaz."
label="Sil"
/>
</form>
</div>
</>
) : (
<p className="text-xs text-muted-foreground">✅ Telegram&apos;a gönderildi, sunucuda saklanmıyor.</p>
)}
<div className="flex items-center gap-2">
<Button asChild variant="outline" size="sm">
<a href={`/api/segments/${segment.id}/video?download=1`}>İndir</a>
</Button>
<form action={deleteRawSegment.bind(null, segment.id)}>
<ConfirmButton
confirmText="Bu segmenti ve içindeki tüm aday klipleri silmek istediğine emin misin? Geri alınamaz."
label="Sil"
/>
</form>
</div>
{segment.candidates.length === 0 ? (
{segment.status === "SENT_TO_TELEGRAM" ? (
<p className="text-sm text-muted-foreground">
Bu segment analiz edilmeden (transkript/9:16 atlanarak) doğrudan Telegram&apos;a gönderildi.
</p>
) : segment.candidates.length === 0 ? (
<p className="text-sm text-muted-foreground">Bu segmentte aday klip bulunamadı.</p>
) : (
segment.candidates.map((c) => (
apps/frontend/app/settings/page.tsx
+30 -1
View File
@@ -6,6 +6,7 @@ import {
updateSttEnabled,
updateAutoRenderEnabled,
updateDeleteAfterTelegramSend,
updateSendRawSegmentsToTelegram,
addCookieProfile,
updateCookieProfile,
deleteCookieProfile,
@@ -36,6 +37,7 @@ export default async function SettingsPage() {
sttSetting,
autoRenderSetting,
deleteAfterSendSetting,
sendRawSetting,
cookieProfiles,
] = await Promise.all([
prisma.appSetting.findUnique({ where: { key: "ytdlp_cookies" } }),
@@ -47,6 +49,7 @@ export default async function SettingsPage() {
prisma.appSetting.findUnique({ where: { key: "stt_enabled" } }),
prisma.appSetting.findUnique({ where: { key: "auto_render_enabled" } }),
prisma.appSetting.findUnique({ where: { key: "delete_after_telegram_send" } }),
prisma.appSetting.findUnique({ where: { key: "send_raw_segments_to_telegram" } }),
prisma.cookieProfile.findMany({ orderBy: { createdAt: "asc" } }),
]);
@@ -60,6 +63,7 @@ export default async function SettingsPage() {
const sttEnabled = sttSetting?.value !== "false";
const autoRenderEnabled = autoRenderSetting?.value !== "false";
const deleteAfterTelegramSend = deleteAfterSendSetting?.value === "true";
const sendRawSegmentsToTelegram = sendRawSetting?.value === "true";
return (
<div className="flex flex-col gap-6">
@@ -259,8 +263,33 @@ export default async function SettingsPage() {
<Card>
<CardHeader>
<CardTitle className="text-sm font-medium text-muted-foreground">Depolama</CardTitle>
<CardTitle className="text-sm font-medium text-muted-foreground">Ham Kayıt Modu</CardTitle>
<CardDescription>
Açarsan her ham segment (analiz/transkript/9:16 zincirine hiç girmeden) tamamlanır tamamlanmaz doğrudan
Telegram&apos;a gönderilip sunucudan silinir — transkript ve 9:16 render tamamen atlanır. Bunu açtıysan
STT ve 9:16 Render ayarlarının kapalı olması mantıklı (aksi halde ikisi de boşa kalır, çünkü bu mod
segmenti analiz kuyruğuna hiç sokmuyor). ~50MB&apos;ı geçen segmentler gönderilemez, sunucuda kalır —
gerekirse yukarıdaki &quot;Sistem Parametreleri&quot;nden segment süresini kısalt.
</CardDescription>
</CardHeader>
<form action={updateSendRawSegmentsToTelegram}>
<CardContent>
<Label className="flex items-center justify-between gap-4 text-sm font-normal">
Ham segmenti doğrudan Telegram&apos;a gönder (analiz atlanır)
<SettingSwitch name="sendRawSegmentsToTelegram" defaultChecked={sendRawSegmentsToTelegram} />
</Label>
</CardContent>
<CardFooter>
<Button type="submit">Kaydet</Button>
</CardFooter>
</form>
</Card>
<Card>
<CardHeader>
<CardTitle className="text-sm font-medium text-muted-foreground">Depolama (9:16 klipler)</CardTitle>
<CardDescription>
Yukarıdaki &quot;Ham Kayıt Modu&quot;ndan farklı — bu, STT+9:16 render zincirinden geçmiş klipler için.
Açarsan her render biten video Telegram&apos;a gönderilip <strong>sunucudan kalıcı olarak
silinir</strong> — sadece Telegram&apos;daki kopyada kalır (geri alınamaz). Gönderim başarısız olursa
dosya sunucuda kalır. Kanal sayısı arttıkça disk doluluğunu önlemek için.
packages/db/prisma/migrations/20260904000000_raw_segment_telegram_send/migration.sql
+5
View File
@@ -0,0 +1,5 @@
-- AlterEnum
ALTER TYPE "RawSegmentStatus" ADD VALUE 'SENT_TO_TELEGRAM';
-- AlterTable
ALTER TABLE "raw_segments" ALTER COLUMN "file_path" DROP NOT NULL;
packages/db/prisma/schema.prisma
+5 -1
View File
@@ -11,6 +11,10 @@ enum RawSegmentStatus {
PENDING
PROCESSED
DISCARDED
// Analiz zincirine (sinyal analizi/STT/9:16) hiç girmeden ham haliyle
// doğrudan Telegram'a gönderildi — file_path bu durumda null (dosya
// gönderim sonrası diskten silindi).
SENT_TO_TELEGRAM
}
enum CandidateSegmentStatus {
@@ -60,7 +64,7 @@ model RawSegment {
id String @id @default(cuid())
sessionId String @map("session_id")
session StreamSession @relation(fields: [sessionId], references: [id])
filePath String @map("file_path")
filePath String? @map("file_path")
duration Int
startedAt DateTime @map("started_at")
status RawSegmentStatus @default(PENDING)