Introduction

What is crtmpserver?

It started as a RTMP server. After a while, it evolved to a generic networking platform capable of handling multiple protocols. This way, RTMP (and friends), became just a fraction of what the platform can do. Here is a far-from-complete list of features (only main ones)

  1. machine independent (x86,amd64,ARM,MIPS,PPC)
  2. operating system independent (linux,*BSD,win32,android)
  3. multiple network protocols support
  4. applications support
  5. RPC and CLI
  6. generic support for multimedia file formats

Requirements

Machine/os requirements

The server can run on one of the following:

  • ARM (linux or android based)
  • x86/amd64 (linux,*BSD,win32,solaris)

Software requirements

To build C++ RTMP Server you need the following:

  • C and C++ compiler
  • Development libraries and header files of OpenSSL (Under Win32 you must use OpenSSL v0.9.x, version 1.0.0 is not supported)

Also, depending on the build method or target platform, you might need one or more of the next software packages:

Compile the server

Linux/BSD

Here are the steps necessary for getting the server up and running. For installing the dependencies please consult this.

  1. check out the source code from svn repository:
    $ svn co --username anonymous https://svn.rtmpd.com/crtmpserver/branches/1.0 crtmpserver

    Hit Enter key when prompted for password (this parameter is empty)

  2. go to the builders directory
    $ cd crtmpserver/builders/cmake
  3. make sure everything is clean by doing
    $ sh cleanup.sh

    Note: This is necessary when updating the sources or you run into troubles like missing libraries reported by cmake

  4. generate the makefile using cmake
    $ cmake .

    Notice the “space dot”

  5. start the actual building process
    $ make

At this point, if no errors occurred, you should have a working binary.

Win32

Make sure that you have administrative privileges before doing the following steps.

  1. Check out source code using SVN. You can use a Subversion client called TortoiseSVN to do this. Put the following values to the TortoiseSVN interface:
    • URL of repository: https://svn.rtmpd.com/crtmpserver/branches/1.0
    • Checkout directory: <Your desired checkout folder>
    • Checkout Depth: Fully recursive
    • Revision: Head revision
    • Username: anonymous
    • Password: <Leave this blank>
  2. When source code has been checked out successfully, go to <checkout folder>/builders/VS2010 folder. Inside that folder, double click the VS2010.sln file.
  3. Optional, build with OpenSSL support:
    1. Download and install “Visual C++ 2008 Redistributables”(Link) and OpenSSL (Link)
    2. If you install OpenSSL to another place than “C:\OpenSSL-Win32”, change OpenSSL include and lib path (in project's properties)
  4. Compile the solution using Microsoft Visual C++.

ARM (linux or android)

Running the server

Linux/BSD

If you are here from previous steps (compiling), you should already be in the right directory which is builders/cmake relative to the directory where the sources are located. Enter the following command:

$ ./crtmpserver/crtmpserver ./crtmpserver/crtmpserver.lua

If you don't see anything happening, that means you either run the daemon mode or something is wrong with the config file (please refer to this page for further details). If everything went well and you are in console mode, you should see something like this:

+-----------------------------------------------------------------------------+
|                                                                     Services|
+---+---------------+-----+-------------------------+-------------------------+
| c |      ip       | port|   protocol stack name   |     application name    |
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 1935|              inboundRtmp|              appselector|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 8081|             inboundRtmps|              appselector|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 8080|             inboundRtmpt|              appselector|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 6666|           inboundLiveFlv|              flvplayback|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 9999|             inboundTcpTs|              flvplayback|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 8989|      inboundHTTPDBAccess|            samplefactory|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 1111|    inboundHttpXmlVariant|                  vptests|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 1112|            inboundTxtCli|                    admin|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 6665|           inboundLiveFlv|             proxypublish|
+---+---------------+-----+-------------------------+-------------------------+
|tcp|        0.0.0.0| 9997|       inboundOggOverHttp|                houseband|
+---+---------------+-----+-------------------------+-------------------------+

Win32

  1. Open a Command Prompt and in the terminal, cd to the checkout directory.
  2. cd to builders\VS2010\Debug. If you compiled the solution in Release mode, cd to builders\VS2010\Release.
  3. Enter this command:
    crtmpserver.exe crtmpserver.lua

ARM (linux or android)

Testing Sample Application

Test RTMP

You can test your compilation by streaming media via RTMP. The application flvplayer is ready for your first rtmp stream.

  1. Put some test media file in the folder /application/flvplayback/mediafolder. For simplicity, put flv files for the meantime.
  2. Open a browser and type http://dl.dropbox.com/u/2918563/flvplayback.swf in the URL to open the sample swf application.
  3. If your server is up, the test media files will be listed in the Available FLVs area.
  4. Double click on a file and wait for playback.

Test RTSP

Stopping the Server

Linux/BSD

From the Console

Press Ctrl+C to send a quit signal. The server is shut down when

Shutting down the logger leaving you in the dark. Bye bye... :(

appears in the console.

When running as Daemon

Kill all crtmpserver processes

Win32

Whether the server is running as daemon or not, you can stop the server by pressing Ctrl+c to send a quit signal.

ARM (linux or android)

Configuration file

The configuration file is actually a lua script which must contain an object called configuration. This will be read by the server and used to fully configure the server. Besides this object called configuration you can have functions, include other lua libraries, etc. In the end, you have to make the configuration object available. The rest of this section will explain the structue of configuration object in great detail. But first, let's take an bird-eye view.

Main structure

configuration=
{
  daemon=false,
  pathSeparator="/",
  logAppenders=
  {
    -- content removed for clarity
  },
  applications=
  {
    -- content removed for clarity
  }
}
configuration structure
key type mandatory description
daemon boolean yes true means the server will start in daemon mode. false means it will start in console mode (nice for development)
pathSeparator string(1) yes This value will be used by the server to compose paths (like media files paths). Examples: on UNIX-like systems this is / while on windows is \. Special care must be taken when you specify this values on windows because \ is an escape sequence for lua so the value should be “\\”
logAppenders object yes Will hold a collection of log appenders. Each of log messages will pass through all the log appenders enumerated here. More details below
applications object yes Will hold a collection of loaded applications. Besides that, it will also hold few other values. More details below

When the server starts, the following sequence of operations is performed:

  1. The configuration file is loaded. Part of the loading process, is the verification. If something is wrong with the syntax please read this
  2. daemon value is read. The server now will either fork to become daemon or continue as is in console mode
  3. logAppenders is read. This is where all log appenders are configured and brought up to running state. Depending on the collection of your log appenders, you may (not) see further log messages
  4. applications is taken into consideration. Up until now, the server doesn't do much. After this stage completes, all the applications are fully functional and the server is online and ready to do stuff

logAppenders

This section contains a list of log appenders. The entire collection of appenders listed in this section is loaded inside the logger at config-time. All log messages will be than passed to all this log appenders. Depending on the log level, an appender may (not) log the message. “Logging” a message means “saving” it on the specified “media” (in the example below we have a console appender and a file).

logAppenders=
{
  {
    name="console appender",
    type="coloredConsole",
    level=6
  },
  {
    name="file appender",
    type="file",
    level=6,
    fileName="/tmp/crtmpserver.log"
  }
},
logAppenders structure
key type mandatory description
name string yes The name of the appender. Is usually used inside pretty print routines
type string yes The type of the appender. It can be console, coloredConsole or file. console and coloredConsole will output to console. The difference between them is that coloredConsole will also apply a color to the message, depending on the log level. Quite useful when eye-balling the console. file log appender will output everything on the specified file
level number yes The log level used. The values are presented just below. Any message having having a log level less or equal to this value will be logged. The rest are discarded. Example: setting level to 0, will only log FATAL errors. Setting it to 3, will only log FATAL, ERROR, WARNING and INFO
fileName string yes If the type of appender is a file, this will contain the path of the file
Log levels
Name Value
0 FATAL
1 ERROR
2 WARNING
3 INFO
4 DEBUG
5 FINE
6 FINEST

Observation: When daemon mode is set to true, all console appenders will be ignored. (Read the explanation for daemon setting here)

applications

This section is where all the applications inside the server are placed. It holds the attributes of each application that the server will use to launch them. Each application may have specific attributes that it requires to execute its own functionality.

applications=
{
  rootDirectory="applications",
 
  {
    -- settings for application 1
    -- content removed for clarity
  },
  {
    -- settings for application 2
    -- content removed for clarity
  },
  {
    -- settings for application 3
    -- content removed for clarity
  }
}
applications structure
key type mandatory description
rootDirectory string true The folder containing applications subfolders. If this path begins with a / or \ (depending on the OS), than is treated as an absolute path. Otherwise is treated as a path relative to the run-time directory (the place where you started the server)

Following the rootDirectory, there is a collection of applications. Each application has its properties contained in an object. See details below

application definition

This is where the settings of an application are defined. We will present only the settings common to all applications. Later on, we will also explain the settings particular to certain applications Since revision 790 there is a new cool feature: mediaStorage; with this feature basicaly an application may have multiple mediaFolder's and .seek/.meta files are now stored into separate folder from media file that are streamed.

{
  name="flvplayback",
  protocol="dynamiclinklibrary",
  description="FLV Playback Sample",
  default=false,
  validateHandshake=true,
  enableCheckBandwidth=true,
  -- this settings are now part of mediaStorage setting
  -- keyframeSeek=true,
  -- seekGranularity=1.5,
  -- clientSideBuffer=12,
  -- generateMetaFiles=true,
  -- renameBadFiles=true,
  aliases=
  {
    "simpleLive",
    "vod",
    "live",
    "WeeklyQuest",
    "SOSample",
    "oflaDemo",
    "chat",
  },
  acceptors = 
  {
    {
      -- acceptor 1
      -- content removed for clarity
    },
    {
      -- acceptor 2
      -- content removed for clarity
    },
    {
      -- acceptor n
      -- content removed for clarity
    },
  },
  -- new feature mediaStorage
  mediaStorage = {
     namedStorage1={
      description="Main storage",
      mediaFolder="/usr/main_storage/media", -- only this parameter IS MANDATORY
      metaFolder="/usr/main_storage/metadata", -- if you have static large file to stream it is good to know that for a file around 500MB 
                                               -- it's .seek file has around 16MB; so it would be preffer to designate metafolder into a system
                                               -- partition which has enough space... for no surprises... :)
      statsFolder="/usr/main_storage/statsFolder",
      enableStats=true,
      clientSideBuffer=16,
      keyframeSeek=false, -- should crtmpdserver DO SEEK ONLY IN key-frame (true/false)? 
                          -- very useful to know in situations like play/pause/resume (meaning pause/seek/play)
      seekGranularity=1,
      generateMetaFiles=false,
      renameBadFiles=false,
    },
    --[[{
      -- here is another example of storage; it does not start with name={...}
       description="Second storage of same application",
       mediaFolder="/usr/second_storage/media/flv",
       metaFolder="/usr/second_storage/metadata",
       statsFolder="/usr/second_storage/statsFolder",
    },]]--
  }, 
  externalStreams = 
  {
    {
      -- stream 1
      -- content removed for clarity
    },
    {
      -- stream 2
      -- content removed for clarity
    },
    {
      -- stream n
      -- content removed for clarity
    },
  },
  authentication=
  {
    -- content removed for clarity
  }
}
application structure
key type mandatory description
name string yes Name of application.
protocol string yes Type of application. The value dynamiclinklibrary means the application is a shared library.
description string no You can put a description of the application here.
default boolean no This flag designates the default application. The default application is responsible in analyzing the connect request and distribute the future connection to the correct application.
validateHandshake boolean no Tells the server to validate the client's handshake before going further. This is optional with a default value of true. If this is true and the handshake fails, the connection is dropped. If this is false, handshake validation will not be enforced and all the connections are accepted no matter if they are correctly hand shaking or not.
keyframeSeek boolean no This instructs the streamer to seek only on key frames. In case of live streaming, this is discarded.
seekGranularity double no The seek resolution/granularity value in seconds. Values are between 0.1 and 600. For example, if granularity is 10 seconds, and a seek to t=2:34 is desired, the seek will actually go to t=2:30. 60 seconds is recommended for full length movies and 1 second for video clips.
clientSideBuffer double no The amount of client side buffer that will be maintained for each connection. Values are between 5 and 30 seconds.
generateMetaFiles boolean no This will generate seek/meta files on application startup.
renameBadFiles boolean no If this flag is true and the media file can't be parsed, the media file will be renamed to *.bad. Otherwise it will be left alone.
aliases object no The application will also be known by this name.
acceptors object no Acceptors hold the service that will be hosted to the server. An application can have its own acceptor, but this is not entirely required, and can be optional.
externalStreams object no
authentication object no
mediaFolder string yes When define mediaStorage this field is mandatory as it points out physical location of media files.
metaFolder string no It holds the location where .seek/.meta files created from files inside mediaFolder are stored.
statsFolder string no Location for stats files.
acceptor structure
key type mandatory description
ip string yes The IP where the service is located. 0.0.0.0 means all interfaces and all IPs.
port string yes Port number that the service will listen to.
protocol string yes The protocol stack handled by the ip:port combination.

Architecture

Layers

Machine layer

Operating System Layer

This layer is composed of libraries with functions that make the entire Evostream ecosystem compatible to any Operating System. OS-specific functions used in the whole solution are made transparent to current Operating System, avoiding any unexpected errors in compiling and running the system. With this layer, Evostream can compile and run on Linux, Windows, Mac, Solaris, and other Operating Systems.

Networking layer

This layer takes care of transporting the data in and out of the server. This layer has the Input/Output Handlers that handles the transfer of data. The network layer implements either of the 3 socket reactors depending on the Operating System:

  • kqueue - BSD based OS
  • epoll - Linux based OS
  • select - win32 based OS

The components layer

Components

Utilities

Buffering

Logging

Lua

Mempool

Misc

Tinyxml

Netio

kqueue

epoll

select

Protocols

These are communication protocols that are interconnected with each other to form a specific way of handling the transfer of data in and out of the server. There are Transport protocols and Endpoint protocols. Transport protocols are those that are used to transfer data further, and all protocol stack must end with an Endpoint protocol. Example is RTMPS with this order of protocols in the stack: [outside_world] ←→ SSL ←→ HTTP ←→ RTMP, where SSL and HTTP are Transport protocols and RTMP is the Endpoint protocol. There should never be another protocol on top of the Endpoint protocol.

Within a protocol stack are Near and Far protocols, each indicating its distance from the server. Near means a protocol that is on the right of a protocol in a stack (means the one closer to the server), whereas Far means the one on the left. In the example protocol stack RTMPS - [outside_world] ←→ SSL ←→ HTTP ←→ RTMP , SSL is the Far protocol of HTTP, and RTMP is its Near protocol of HTTP. The protocol that is nearest to the server is the Near-endpoint (RTMP in this case), and the one closest to the outside world is the Far-enpoint (SSL in this case).

Base Protocol

The BaseProtocol class is the base class on which all protocols derive from. This is class is

Base Protocol Factory

The BaseProtocolFactory class in the base class on which all protocol factory derive from.

Protocol Factory

Evostream has a “bank” of protocols and protocol stacks that are all available applications to use. A BaseProtocolFactory class is used as a base class of all protocol factories. Atomic protocols are declared and protocol chains are spawned in the factory. Resources from the factory will then be exported for use by the applications.

Protocol Manager

Applications and protocol handlers

Streaming

Media formats

flv

mkv

mp3

mp4

Interoperability

RPC

CLI

My first application

Customize Configuration File

Create a custom protocol and a protocol factory

Extending media file formats collection

Use Evostream as Client (Test it against itself)

Troubleshooting

  1. The configuration file got messed up so the server does not run correctly.
    • You can double check the syntax of your configuration file for errors. It could take only one wrong syntax to cause runtime error.
    • If you want to get the original copy of the configuration file from the repository, just delete your configuration file then update the source code. Don't forget to compile the source code after updating it.
  2. Media files are not streaming.
    • Make sure your server is running with no errors.
    • Meta and Seek files may be broken due to previous unexpected errors. Delete these files and try streaming your media again.
  3. Evostream on Mac OSX is no part of this documentation.
    • Evostream is readily available on Mac OSX platform. Compiling, Running, and Stopping the server on your Mac can be done by following instructions on Linux/BSD.
  4. On Microsoft Visual C++, errors in compilation indicates missing libraries or duplicate classes.
    • Since the source code is updated regularly, there might be new definitions that need to be added on the project properties. To check these definitions, open the CMakeLists.txt located at /builders/cmake/ folder. Make sure that the pre-processor definitions in your project has the ones inside the ADD_DEFINITIONS tag in the CMakeLists.txt. Don't forget to clean and build the solution.

Extentions

Adding Gearman protocol support. here

Example Setup's

"Live" streaming of a media file

This tutorial explains how to “live stream” a media file using ffmpeg and rtmpd. See here for details.

A Powerful Live Streaming Setup

Can the server be used in load-balancing mode? Yes, it can. More details here

A Full CentOS Build-Up

Here you can find details about how to build and run everything on CentOS

Protecting your streams from webpage copy&paste flash code, listing or recording

This is where you can find details of how to protect your streams. (by Rani)

Basic setup application with new feature mediaStorage

This is where you can find details of how to manage a very basic and powerful streaming scenario with only one application customized and tuned for real-world media streaming. (by Rani)

Basic setup fully-functional secure application

Here is where you can find details of how to manage a very basic and powerful streaming scenario with only one application customized and tuned for real-world media streaming protected by a few builtin security features. (by Rani)

documentation.txt · Last modified: 2013/03/06 16:09 by rani
 
Except where otherwise noted, content on this wiki is licensed under the following license: GNU Free Documentation License 1.3
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki