please dont rip this site

BOSH: Bidirectional Operation over Synchronous Html Remote Data Transfer Method

Starting from a standard node / express server:
const express = require('express')
const http = require('http')
const WebSocket = require('ws') //
const bodyParser = require( "body-parser") //parse post body adata
const bcryptjs = require('bcryptjs') //to encrypt passwords for storage

const app = express()
const PORT = 8080
For node.js on the app engine, there are two environments: Standard and Flexible.

To reduce ongoing cost of operation, we are starting with the standard environment. Sadly, that does not support websockets, which would be very easy to use, but, luckily Jitsi taught us about BOSC

BOSH supports server push notifications by simply not responding to a request from the client until the server has something to send to the client. The request simply hangs until data arrives (via another connection to the server from the sender) or until it times out. In either case, it is the clients responsibility to re-establish the request as quickly as possible and so continue listening for data from the server, or more accurately from a sender via the server.

The other issue with the Standard Environment is that it can be shut down and started up at Googles will. This makes keeping session state interesting. You can log in, get a cookie, and if that cookie was stored in a local RAM session store, it can be forgotten at any moment. Luckily, google/@datastore supports using a Cloud Firestore in Datastore mode as the store.

const {Datastore} = require('@google-cloud/datastore');
const session = require('express-session') //session mgmt
const DatastoreStore = require('@google-cloud/connect-datastore')(session);
const data_store = new Datastore({
      // @google-cloud/datastore looks for GCLOUD_PROJECT env var. Or pass a project ID here:
      // projectId: process.env.GCLOUD_PROJECT,
      projectId: APP_ENGINE_PROJECT_ID,
      // @google-cloud/datastore looks for GOOGLE_APPLICATION_CREDENTIALS env var. Or pass path to your key file here:
      keyFilename: process.env.GOOGLE_APPLICATION_CREDENTIALS

//Session setup seperate sessionParser so 
  store: new DatastoreStore({
    kind: 'express-sessions', 
    // Optional: expire session after exp milliseconds. 0 means do not expire
    // note: datastore doesnt auto del expired sessions. Run separate cleanup req's to remove expired sessions
    expirationMs: 0,
    dataset: data_store
  resave: false,
  saveUninitialized: false,
  secret: 'Like id tell you'
The only problem with that is that cookies are perminant and do not expire, even if you tell them to. It's necessary to manually clear out the database once in a while

//just serve up static files if they exist
app.use(bodyParser.urlencoded({extended: true}))

/* Authentication section removed, it basically just sets a user name
req.session.user = username
Rather than use available BOSH packages (which were poorly document and very confusing) a microscopic version of BOSH was included. For this service, we can be relatively certain that the engine will not be shut down, because it's always pending a BOSHout request. As long as one listener is listening, google doesn't have time to shut down the engine and therefore drop the boshs array.

Operation: BOSHout requests come in, get added to the boshs array, and then nothing else is done. There is no reply, no error, no nothing. This causes the socket to stay open, as the client waits for the server to respond. Next, a BOSHin request comes in, with a "to" parameter for the user who made the BOSHout request. Since that user is found in the boshs array, the socket response object can be found, and the message sent to that client. The entry in the boshs array is then deleted so it won't be found by another BOSHin until the BOSHout client re-establishes the connection.

var boshs = {}
BOSHtimeout = 30000 //in milliseconds

app.get('/BOSHout', function (req, res, next) { 
  console.log("BOSHout,  user:"+req.session.user)
  if (!req.session.user) {
    var err = new Error('FAIL: Login')
    err.status = 403
    next(err) //skips to error handler. 
        //console.log('BOSHout timeout '+req.session.user)
        res.send('BOSHout timeout')
        delete boshs[req.session.user]
    }, BOSHtimeout)
//dont reply or end. 

app.get('/BOSHin', function(req,res, next) {
    //Check that the requestor is logged in. 
    if (!req.session.user) {
        var err = new Error('FAIL: Login')
        err.status = 403
        next(err) //skips to error handler. 
    let msg = req.query.msg || "none"
    let to =
    let bohc = {}
    //parameter is the session.user to send the message to
    if (to) { //console.log("to "+to)
        if ( boshs[to]) { console.log("found")
            bosh = boshs[to] //lookup session id from username
    bosh.stat = "unknown"
    if ( { //console.log("clearing timeout")
    if (bosh.res) { //console.log("responding")
        bosh.res.setHeader("from_user", req.session.user)
        bosh.res.send('BOSHin message:' + msg) //don't change w/o Fry.
        bosh.stat = bosh.res.headersSent ? "good" : "bad" 
        //bosh.res = undefined //make sure we know this response object is used
        delete boshs[to]
    console.log("to:"". stat:"+bosh.stat+". ")
    res.send('BOSHin to:' + ". Status:" + bosh.stat) 
If the user requested in the BOSHin is not found in the boshs array, "unknown" is returned to the BOSHin client. Otherwise, "good" is returned.

The rest of the code is just the standard node / express setup:

// If we get here, the file wasn't found. Catch 404 and forward to error handler
app.use(function (req, res, next) {
  var err = new Error('File Not Found');
  err.status = 404;

// Error handler define as the last app.use callback
app.use(function (err, req, res, next) {
  res.status(err.status || 500)

Support for this server is being added to DDE via the Messaging class.

Testing shows throughput rates of 50ms and latency of 80ms (which is amazing) to 300ms under normal use.

All code archived here:

You can see the result here:
(This is without latency compensation of any type. We got 60ms round trips regularly, with as much as 500ms delays from time to time. The major issue was actually the compression and transmission of the return video. Reducing it's resolution and framerate greatly improved teleoperation.)

file: /Techref/method/BOSH.htm, 8KB, , updated: 2023/9/16 10:32, local time: 2024/2/23 22:59, owner: JMN-EFP-786,

 ©2024 These pages are served without commercial sponsorship. (No popup ads, etc...).Bandwidth abuse increases hosting cost forcing sponsorship or shutdown. This server aggressively defends against automated copying for any reason including offline viewing, duplication, etc... Please respect this requirement and DO NOT RIP THIS SITE. Questions?
Please DO link to this page! Digg it! / MAKE!

<A HREF=""> BOSH: Bidirectional Operation over Synchronous Html Data Transfer Method</A>

After you find an appropriate page, you are invited to your to this massmind site! (posts will be visible only to you before review) Just type a nice message (short messages are blocked as spam) in the box and press the Post button. (HTML welcomed, but not the <A tag: Instead, use the link box to link to another page. A tutorial is available Members can login to post directly, become page editors, and be credited for their posts.

Link? Put it here: 
if you want a response, please enter your email address: 
Attn spammers: All posts are reviewed before being made visible to anyone other than the poster.
Did you find what you needed?


Welcome to!

Site supported by
sales, advertizing,
& kind contributors
just like you!

Please don't rip/copy
(here's why

Copies of the site on CD
are available at minimal cost.

Welcome to!