Implementation Techniques for Multimedia Storage Servers and Networking (Spring 1999)

Tzi-cker Chiueh

Group members

Susanne Cech 9560316 Michael Kropfberger 9555885
Johannes Kosche 9560368 Hannes Tschofenig 9460247


Table of Contents

Project Goal
Design Decisions
Implementation Details
Performance Analysis

Project Goal

Implementation of a user-level real-time network storage server with the following specifications:


The main idea of a real-time network storage server is to deliver files at a specified rate to a number of clients. Traditionally, a client-server architecture is used.

Our server consists of several parts, such as a disk storage subsystem and a control unit, which is responsible for the admission control and the scheduling of the request.  The admission control module decides if the server can satisfy a new request for the entire connection.  Cyclic scheduling means that within one cycle the required media blocks of all streams are retrieved from disk and sent to the clients.

Constraints of a user-level real-time network storage server:

Design Decisions

1. Server Push vs Client Pull Model
For our purpose, a server push model seemed to be well suited, because it simplifies the admission control implementation.

2. Admission Control
Our admission control is based on statistical averaging. To be able to decide whether to allow a new request to join or not is based on measured time consumtions of acutal fullfilled requests of past cycles.  The basic assumption behind this averaging is that we use historic calculated requirements to predict the future requirements.

To be more formallly, the following equation must be satisfied:

R = R1 + R2 + ... +Rn + O < P (I/O cycle) Ri.. time  for request i per I/O cycle

O.. additional overhead

P.. maximal time bound for one I/O cycle

Figure 1. Ideal cycle.

If all requests are satisfied within the current cycle and time is left, there are two possibilities to deal with this problem: Either the server waits in a loop (busy waiting), which is CPU intensive, or it sleeps until the end of the cycle. The operating system is responsible for waking the server, which cannot be that exact. As a consequence, the process might sleep longer as required [Figure 2] and therefore the next cycle-time is shortened by the time it slept too long[Figure 3].

Figure 2. Process sleeps too long.

Figure 3. Thus, next cycle is shorter.

Implementation Details


Used data structures:

typedef struct mm_client {
  struct sockaddr_in  client;        // socket address
  char                hostname[STRLEN]; // hostname for convenience
  int                 port;           // port number for convenience
  char                file[STRLEN];     // requested file
  FILE                *fp;            // file pointer
  long                fpos;           // starting offset within file
  long                rate;           // rate in bytes per second
  long                usecs;             // time of last serve
  long                avg_usecs;        // average time of previous serves
  struct mm_client    *prev;
  struct mm_client    *next;
} mm_client;
Listing 1.  List structure for client requests

typedef struct mm_stats {
  float  avg_bytes_per_usec; // average bytes per usec over the server's life time
  long   sum_usecs;         // sum of all usecs used for clients in this I/O cycle
  long   avg_sum_usecs;     // average sum of usecs used for clients over the server's life time
  long   sum_bytes;         // sum of all bytes sent in this I/O cycle
  int    num_clients;       // number of concurrent clients
  long   remaining_time;    // time left in actual I/O cycle
  long   adjust;            // time server slept too long
  long   avg_adjust;        // average of adjusts
  long   algo_time;         // time used for general code parts
  long   cycles;            // number of cycles
  long   working_time;      // time  spent on serving all clients in this cycle
  long   sum_working_time;  // sum of all working times spent over the server's life time
} mm_stats;
Listing 2.  Data structure for statistics.

Main server layout:



Performance Analysis

Test Environment

Operating system:
Linux 2.0.36 Kernel
RAM: 256MB
SCSI Controller: Adaptec AHA2940
Network: Intel EtherExpress Pro (100MBit)
always requested a rate of 1.5 MBit/sec
All those tests imply a completely unused network and close-to-idle CPUs.
Other network load would increase collisions and thus influence the time needed to serve a request.
Also a high CPU usage of other concurrent processes (and most of all concurrent file access) would increase the time needed to serve a request.

Test Results

Figure 4 shows how the number of accepted clients decreases when distinct files are requested for streaming.

Figure 4. variable client requests and increasing number of distinct files

Figure 5 shows fixed client requests (45 clients)  and increasing distinct files.
The increasing number of distinct files leads to an increase of microsecs spent on each client to fulfill the wanted rate.

Figure 5. fixed client requests (45 clients)  and increasing number of distinct files