Skip to content

Instantly share code, notes, and snippets.

@kevinGodell

kevinGodell/raspberry_pi_4_model_b_rev_1_1_4gb_ffmpeg.md

Created August 17, 2020 18:06
Show Gist options
  • Save kevinGodell/43bd6d075d15bca6ac5a1c1c11f4e947 to your computer and use it in GitHub Desktop.
Save kevinGodell/43bd6d075d15bca6ac5a1c1c11f4e947 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. kevinGodell created this gist Aug 17, 2020.
    236 changes: 236 additions & 0 deletions raspberry_pi_4_model_b_rev_1_1_4gb_ffmpeg.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,236 @@
    # Raspberry Pi 4 Model B Rev 1.1 (4GB)
    ## ffmpeg hardware decoding experiment
    ### modifications (needed for hardware decoding multiple video streams)
    - gpu_mem=512 was set in /boot/config.txt

    ### dependencies
    - [mp4frag](https://www.npmjs.com/package/mp4frag) (used for parsing the piped mp4 from ffmpeg)
    - [pipe2jpeg](https://www.npmjs.com/package/pipe2jpeg) (used for parsing the piped jpeg from ffmpeg)

    ### description
    - copies main and sub streams from rtsp ip cam and muxes to individual fragmented mp4
    - h264_mmal codec used for decoding rtsp sub stream
    - encodes full sized and downscaled jpegs from rtsp sub stream

    ### results
    - uses ~ 10% cpu load (will vary based on input video sizes and output jpeg sizes and framerate)

    ### source
    ```js
    'use strict';

    const { spawn } = require('child_process');

    const M4F = require('mp4frag');

    const P2J = require('pipe2jpeg');

    const mp4fragMain = new M4F();

    const mp4fragSub = new M4F();

    const pipe2jpegFull = new P2J();

    const pipe2jpegDownscaled = new P2J();

    let segmentCounterMain = 0;

    let segmentCounterSub = 0;

    let jpegCounterFull = 0;

    let jpegCounterDownscaled = 0;

    mp4fragMain.on('segment', (data) => {
    console.log('segment main', ++segmentCounterMain, data.length);
    });

    mp4fragSub.on('segment', (data) => {
    console.log('segment sub', ++segmentCounterSub, data.length);
    });

    pipe2jpegFull.on('jpeg', (data) => {
    console.log('jpeg full', ++jpegCounterFull, data.length);
    });

    pipe2jpegDownscaled.on('jpeg', (data) => {
    console.log('jpeg downscaled', ++jpegCounterDownscaled, data.length);
    });

    const params = [
    '-hide_banner',

    '-progress',
    'pipe:1',

    '-loglevel',
    'quiet',

    '-hwaccel',
    'rpi',

    '-c:v',
    'h264_mmal',

    /*
    1st input, rtsp sub stream
    */

    // format rtsp
    '-f',
    'rtsp',

    // use tcp (some ip cams can only use udp)
    '-rtsp_transport',
    'tcp',

    // input ip cam
    '-i',
    'rtsp://192.168.1.4:554/user=admin_password=pass_channel=1_stream=1.sdp',

    /*
    2nd input, rtsp main stream
    */

    // format rtsp
    '-f',
    'rtsp',

    // use tcp (some ip cams can only use udp)
    '-rtsp_transport',
    'tcp',

    // input ip cam
    '-i',
    'rtsp://192.168.1.4:554/user=admin_password=pass_channel=1_stream=0.sdp',

    /*
    create filter that will use sub stream to create jpegs (full and scaled), throttled to 1 fps
    1. use 1st input [0:v:0]
    2. throttle fps [throttled]
    3. split output [full][split]
    4. down scale [downscaled]
    */
    '-filter_complex',
    '[0:v:0] fps=fps=1 [throttled]; [throttled] split=2 [full][split]; [split] scale=iw/3:-1:flags=bicubic:out_range=jpeg [downscaled]',

    /*
    1st output, rtsp sub stream copied and muxed to fragmented mp4
    */

    // map 1st input's video
    '-map',
    '0:v:0',

    // no audio
    '-an',

    // video codec copy(no encoding)
    '-c:v',
    'copy',

    // format mp4
    '-f',
    'mp4',

    // make mp4 cross-browser compatible with media source extension
    '-movflags',
    '+frag_keyframe+empty_moov+default_base_moof',

    // send to pipe:3, will be received by ffmpeg.stdio[3].pipe(mp4fragSub)
    'pipe:3',

    /*
    2nd output, full sized jpeg
    */

    // map full sized jpeg
    '-map',
    '[full]',

    // no audio
    '-an',

    // video codec mjpeg
    '-c:v',
    'mjpeg',

    // video quality
    '-q:v',
    '15',

    // format image2pipe
    '-f',
    'image2pipe',

    // send to pipe:4, wil be received by ffmpeg.stdio[4].pipe(pipe2jpegFull)
    'pipe:4',

    /*
    3rd output, down scaled jpeg
    */

    // map down scaled jpeg
    '-map',
    '[downscaled]',

    // no audio
    '-an',

    // video codec mjpeg
    '-c:v',
    'mjpeg',

    // video quality
    '-q:v',
    '15',

    // format image2pipe
    '-f',
    'image2pipe',

    // send to pipe:5, wil be received by ffmpeg.stdio[5].pipe(pipe2jpegDownscaled)
    'pipe:5',

    /*
    4th output, rtsp main stream copied and muxed to fragmented mp4
    */

    // map 2nd input's video
    '-map',
    '1:v:0',

    // no audio
    '-an',

    // video codec copy
    '-c:v',
    'copy',

    // format mp4
    '-f',
    'mp4',

    // make mp4 cross-browser compatible with media source extension
    '-movflags',
    '+frag_keyframe+empty_moov+default_base_moof',

    // send to pipe:6, will be received by ffmpeg.stdio[6].pipe(mp4fragMain)
    'pipe:6',
    ];

    const ffmpeg = spawn('ffmpeg', params, { stdio: ['ignore', 'pipe', 'inherit', 'pipe', 'pipe', 'pipe', 'pipe'] });

    ffmpeg.stdio[1].on('data', (data) => {
    console.log(`\npipe:1\n${data.toString()}`);
    });

    ffmpeg.stdio[3].pipe(mp4fragSub);

    ffmpeg.stdio[4].pipe(pipe2jpegFull);

    ffmpeg.stdio[5].pipe(pipe2jpegDownscaled);

    ffmpeg.stdio[6].pipe(mp4fragMain);

    ```