File upload

Uploading a file is an essential function of GoSpeech.

📘
Note
Read more about Supported File Formats and Supported Languages on the respective pages.

If it's suitable for your system we suggest to use our npm package, which is dsigned to handle the upload in the easiest way possible. Please contact us via Enterprise/API request form on the Billing plan page to get access to our internal npm repository.

If you can't use npm package please see the section Manual upload.

Using npm package

Add the following lines to your project's .npmrc:

registry=https://nexus.dev.app.gospeech.com/repository/gs-npm/
always-auth=true

Then run npm install gospeech-upload-sdk.

Import uploadApi (as default) and all necessary types (i.eRecord, UploadOptions, etc. ) from gospeech-upload-sdk, configure options and record and use it as shown below:

const result = await uploadApi.upload(options, record, file);

📘
Note
In the createdRecordyou can track the field progressBytes to keep up to date with the progress of the upload. Using the expression progressBytes / fileSize * 100 it is possivle to see the current progress in percent.

The result has the type UploadResponse. It may contain the error type UploadResponseError:

interface UploadResponse {
  success: boolean;
  error?: UploadResponseError;
  recordId?: string;
}

class UploadResponseError {
  readonly title?: string;
  readonly detail: string;
  readonly status: HttpStatus;
  readonly extensions?: {
    codes: LimitRecordCode[]
  };
}

If the result is successful, use recordId for further work. Otherwise you will find more information in the error codes. The codes field in the error.extensions can contain from none to several specific error codes. For more information on the meaning of these codes please see validation result.

Sample app to use npm package

Please contact us via the Enterprise/API request form on the Billing plan page to receive the Angular sample app that uses the npm package to upload the file.

Run npm install to install all packages and ng serve to build and run the app.

You can finde the code related to the upload in UploadComponent under the path src\app\components\upload\upload.component.ts. You have to setup accessToken, applicationUrl and authorizationUrl. Also you can change the Record settings, which are used in

const result = await uploadApi.upload(options, record, file);

When you start the sample app and open it in the browser you should click on the "Upload" button and select the media file to be uploaded. Next, you can monitor the sending of upload requests in the browser developer tools on the network tab. Eventually, you can see the result of the upload in the console: the result status (successful or not) and the record id of the uploaded data or the error details if something went wrong.

Manual upload

If you can't use the npm package, you should perform the following steps to upload a file:

[Optional] check the recognition services availability with API.
Validate the file for the upload with API.
Configure the create Record request:

{
	"name": "Info",
	"title": "Info",
	"numberReference": "",
	"producer": "",
	"priority": false,
	"recognitionOptions": {
		"language": "en-us",
		"dictionariesIds": [],
		"multipleSpeakers": true,
	},
	"fileSize": 49428885,
	"trackTimeMilliseconds": 505545,
	"fileName": "Info.mp4"
}

📘
Note
You can receive all the supported recognition languages from API and the available dictionaries from API.

Create record with API.
Upload the file parts using partial hashes in cycles until the entire file is uploaded with API.
Set the status Uploaded for the record to complete the upload with API.

Below you will find a sample of a file upload process written in pseudocode, typescript and c#:

function upload(appOptions, recordOptions, file) 
{
  recordRequest = createRequest(file, recordOptions);

  callValidateApi(options, recordRequest.trackTimeMilliseconds, recordRequest.fileSize);

  createdRecordData = callCreateRecordApi(options, recordRequest);

  finalHash = uploadFilePart(options, file, createdRecordData.multipartUploadId, recordOptions);;

  hashBase64 = convertToBase64(finalHash);
  callSetUploadedStatusApi(options, createdRecordData.id, hashBase64);
  
  return createdRecordData.id;
}

function uploadFilePart(options, file, id, recordOptions) 
{
  fileSize = file.size;
  blockSize = 1024 * 1024;
  hashLength = 128;
  hasher = createBLAKE2b(hashLength);

  for (offset = 0; offset < fileSize; offset += blockSize) {
    partBlob = readFileDataFromTo(file, offset, minimumOf(offset + blockSize, fileSize));
    partData = convertToUint8Array(partBlob);
    updateHasher(hasher, partData);
    partHash = computeBlake2b(partData, hashLength);
    partUpload = createRequest(id, offset, partHash);

    callUploadFilePartApi(options, partUpload, partData);

    recordOptions.progressBytes = offset;
  }

  recordOptions.progressBytes = fileSize;
  return hasher;
}

import { blake2b, createBLAKE2b, IHasher } from "hash-wasm";
import { Buffer } from 'buffer';
import {
  createRecord,
  setUploadedStatus,
  uploadFilePart,
  validate,
} from "./api.service";

const upload = async (uploadOptions: UploadOptions, record: Record, file: File)
  : Promise<string> => {
  record.progressBytes = 0;

  const recordRequest = await getRecordCreationRequest(file, record);

  await validate(uploadOptions, recordRequest.trackTimeMilliseconds, recordRequest.fileSize);

  const createdRecordData = await createRecord(uploadOptions, recordRequest);

  const hasher = await uploadFile(uploadOptions, file, createdRecordData.multipartUploadId, record);;

  const hashHEX = hasher.digest();
  const hashBase64 = Buffer.from(hashHEX, 'hex').toString('base64');
  await setUploadedStatus(uploadOptions, createdRecordData.id, hashBase64);
  
  return createdRecordData.id;
}

const uploadFile = async (uploadOptions: UploadOptions, file: File, id: string, record: Record)
  : Promise<IHasher> => {
  const fileSize = file.size;
  const blockSize = 1024 * 1024;
  const hashLength = 128;
  const hasher = await createBLAKE2b(hashLength);
  hasher.init();

  for (let offset = 0; offset < fileSize; offset += blockSize) {
    const partBlob = file.slice(offset, Math.min(offset + blockSize, fileSize));
    const partData = new Uint8Array(await partBlob.arrayBuffer());
    hasher.update(partData);
    const partHash = await blake2b(partData, hashLength);
    const partUpload = { id, offset, partHash };

    await uploadFilePart(uploadOptions, partUpload, partData);

    record.progressBytes = offset;
  }

  record.progressBytes = fileSize;
  return hasher;
}

const getRecordCreationRequest = async (file: File, record: Record)
  : Promise<CreateRecordRequest> => {
  const request = {
    ...record,
    fileName: file.name,
    fileSize: file.size,
    isFileEncrypted: false,
    numberReference: '',
    priority: false,
    producer: '',
  } as CreateRecordRequest;
    
  new Audio(URL.createObjectURL(file)).onloadedmetadata = (e: any): void => {
    const duration = +(e.currentTarget.duration * 1000).toFixed();
    request.trackTimeMilliseconds = duration;
  };
  return request;
}

using Blake2Fast;
using System;
using System.Buffers;
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

internal class RecordService : ServiceBase, IRecordService
{
    public async Task<Guid> Upload(string filePath, GoSpeechUploadOptions options, string managedUsername = null, CancellationToken token = default)
    {
        var createResponse = await CreateRecord(filePath, options, managedUsername, token);

        var finalHash = await UploadFileInParts(filePath, createResponse.MultipartUploadId, managedUsername, token);

        await SetUploadedStatus(createResponse.Id, finalHash);

        return createResponse.Id;
    }

    private async Task<CreateRecordResponse> CreateRecord(string filePath, GoSpeechUploadOptions options, string managedUsername = null, CancellationToken token = default)
    {
        var request = new CreateRecordRequest()
        {
            FileName = options.OriginalFileName ?? Path.GetFileName(filePath),
            FileSize = new FileInfo(filePath).Length,
            Title = options.Title,
            Name = options.RecordName,
            NumberReference = string.Empty,
            Priority = options.Priority,
            Producer = string.Empty
        };

        return await ApiPostAsync<CreateRecordResponse, CreateRecordRequest>(GoSpeechRequestMessage.CreateActionUrl("/records", managedUsername), request, token);
    }

    private async Task<byte[]> UploadFileInParts(string filePath, Guid multipartUploadId, string managedUsername = null, CancellationToken token = default)
    {
        const int DIGEST_LENGTH = 16;
        const int BUFFER_LENGTH = 1024 * 1024;

        var hasher = Blake2b.CreateIncrementalHasher(DIGEST_LENGTH);
        var buffer = ArrayPool<byte>.Shared.Rent(BUFFER_LENGTH);
        int bytesRead = 0;
        int offset = 0;

        using (var data = File.Open(filePath, FileMode.Open))
        {
            while ((bytesRead = await data.ReadAsync(buffer, 0, buffer.Length, token)) > 0)
            {
                hasher.Update(buffer.AsSpan(0, bytesRead));
                await UploadFilePart();

                offset += bytesRead;
            }
        }

        return hasher.Finish();

        async Task UploadFilePart()
        {
            var partHash = Blake2b.ComputeHash(DIGEST_LENGTH, buffer.AsSpan(0, bytesRead));

            var headers = new IHeader[]
            {
                new SimpleHeader("x-gbs-id", multipartUploadId.ToString()),
                new SimpleHeader("x-gbs-offset", offset.ToString()),
                new SimpleHeader("x-gbs-part-hash-blake2b-128-hex", string.Concat(Array.ConvertAll(partHash, x => x.ToString("X2")))),
            };

            var response = await ApiPostAsync<PartUploadResult, ByteArrayContent>(
                GoSpeechRequestMessage.CreateActionUrl("/upload", managedUsername),
                new ByteArrayContent(buffer, 0, bytesRead), token, new HeaderCollection(headers));
        }
    }

    private async Task SetUploadedStatus(Guid recordId, byte[] finalHash, string managedUsername = null, CancellationToken token = default)
    {
        var request = new CompleteRecordUploadRequest()
        {
            Hash = Convert.ToBase64String(finalHash)
        };

        _ = await ApiPatchAsync<string, CompleteRecordUploadRequest>(
            GoSpeechRequestMessage.CreateActionUrl($"/records/{recordId}/status/uploaded", managedUsername),
            request, token);
    }
}