JSON Alternative CBOT in Python — Feedback Needed

Python Help
1

Hi!

I have created a protocol called CBOT (Character Based Object Transport) which is a direct JSON replacement especially for web development. Basically, it is a better and more robust protocol for transporting data while not being a binary protocol. The protocol already has full support in Java and TypeScript/JavaScript.

However, I have wanted to expand the language support and Python was a natural next choice for that. I am not really a Python expert but I have been able to get the implementation to a point where most of the protocol’s capabilities actually work. And I would really appreciate any feedback or live testing for the implementation.

So far I have:

  • Implementation that supports most of the protocol capabilities

But I am not sure about:

  • Whether I am using Python’s different datatypes correctly
  • If the code is done in a pythonic way
  • And many other things as well…

Here is a simple example of how the library usage looks in Python and JavaScript when it is used as a pure JSON replacement without schema:

from sisujs_cbot import Cbot, CbotOptions

# Create a CBot instance
cbot = Cbot(CbotOptions())

# Data
data = {
  "name": "John Smith",
  "age": 41.0,
  "address": {
    "street": "Second Avenue",
    "postalCode": "1356-A",
    "city": "Yorkistan"
  },
  "isNiceGuy": True,
  "hobbies": [
    "Playing cards",
    "Shopping",
    "Asking odd questions"
  ]
}

# Serialize
message = cbot.serialize(data)

# Deserialize
deserialized = cbot.deserialize(message)

Alternatively, the same in JavaScript:

import { Cbot } from "@sisujs/meta-cbot";

// Create a Cbot instance
const cbot = Cbot.getInstance();

// Data
data = {
  name: "John Smith",
  age: 41.0,
  address: {
    street: "Second Avenue",
    postalCode: "1356-A",
    city: "Yorkistan"
  },
  isNiceGuy: true,
  hobbies: [
    "Playing cards",
    "Shopping",
    "Asking odd questions"
  ]
};

// Serialize
const message = cbot.serialize(data);

// Deserialize
const deserialized = cbot.deserialize(message);

The protocol also has support for using schema (or typed model in my terminology), which creates a protocol-level contract between parties. And one feature that differentiates this protocol from others is that it has well-established support for polymorphic types.

Relevant links for the project and the implementation itself:

If you are interested in the typed model examples, there are ongoing validation tests for all three languages to demonstrate how they match models with each other:

Thank you for your help.

2

All or none for a new standard please. stops you from having something that works in one area but won’t work when exported to another. I have seen that problem before and had to carefully understand what part of the “standard” was portable. One of the bigexamples was in browser dependent extensions to HTML back in the day when Microsoft tried to lock you into Internet Explorer.

3

Naturally. My intent was the hope that I could get some insight whether chosen patterns or naming conventions are usable before I make my first release. It is not about making incomplete releases.

Regardless, first release is quite close already so everything begins to look ready.

4

What’s better about it? How is it “more robust”? Does it support user defined types? Does it serialize trees or object graphs?

The API looks very clean from your example.

As for how pythonic your code is, well ask yourself: How many places would you have to modify for adding support for say, a Complex128 type?

Small stylistic choice, I would avoid the line continuation character in the builder patterns. Instead use parentheses to escape Python’s strict indentation rules:

(
    Validator
    .begin(cbot, map)
    .validate()
    .validate_deserialize("basic_map/mixed.msg")
)
5

I’m glad you asked.

When I began this process many years ago, I had two major problems using JSON to transport data: it supports only a few types and it does not have any concept of schema. Especially, I wanted a protocol that understands classes and polymorphism, and to be character based. I wanted it to become kind of invisible transport medium where you do not need to worry about what you are transporting. It just works.

For native types, this protocol supports:

  • 32-bit, 64-bit, and arbitrary precision integers
  • 32-bit, 64-bit, and arbitrary precision floats
  • Strings, booleans
  • Date types: zone datetime, local datetime, local date, and time
  • Byte arrays
  • Collection types: array/list, set, and map

The rest of the user-defined types can be modeled through schemas (or typed model in my terminology). This protocol serializes object trees. The serialized message format is not actually human-readable, but it can be visualized in an assembly-like format.

For classes and polymorphism support, here is a silly but working example:

from datetime import date

from pydantic import BaseModel

from sisujs_cbot import (
  Cbot,
  CbotNamespace,
  CbotOptions,
  int32_type,
  local_date_type,
  model_object_type,
)


ns = CbotNamespace("animals")

class AnimalBase(BaseModel):
    id: int = int32_type()

class CanineBase(AnimalBase):
    pass

class FelineBase(AnimalBase):
    pass

@ns.model()
class Dog(CanineBase):
    pass

@ns.model()
class Fox(CanineBase):
    pass

@ns.model()
class Cat(FelineBase):
    agility_wins: list[date] = local_date_type()
    
@ns.model()
class Tiger(FelineBase):
    pass

@ns.model()
class Earth(BaseModel):
    animals: list[AnimalBase] = model_object_type() # Any animal will do

ns.seal()

cbot = Cbot(CbotOptions(namespaces=[ns]))

animals: list[AnimalBase] = [
    Dog(id=1),
    Cat(id=2, agility_wins=[date(2025, 3, 4), date(2026, 2, 13)]),
    Fox(id=3),
    Dog(id=4),
    Tiger(id=5)
]

animals2: list[AnimalBase] = [
    Dog(id=1),
    Cat(id=2, agility_wins=[date(2025, 3, 4), date(2026, 2, 13)]),
    Dog(id=4), # Fox and Dog changed places
    Fox(id=3),
    Tiger(id=5)
]

earth = Earth(animals=animals)

msg_animals = cbot.serialize(animals)
msg_earth = cbot.serialize(earth)

result_animals = cbot.deserialize(msg_animals)
result_earth = cbot.deserialize(msg_earth)

print(msg_earth)
print(cbot.visualize(msg_earth))

assert animals == result_animals
assert animals2 != result_animals
assert earth == result_earth

The key point is at the end where the asserts prove that you get the same animal types in the correct order without any external encoding.

And about the message itself, print(msg_earth) looks like:

1123ed836
E   $
B   !C
E   #
B   'Ia1
F
E   "
B    C
Ij2025-03-04
Ij2026-02-13
D
B   'Ia2
F
E   %
B   'Ia3
F
E   #
B   'Ia4
F
E   &
B   'Ia5
F
D
F

And print(cbot.visualize(msg_earth)) is:

MCSM 123ed836
OBJB 4 (animals.Earth)
  ASGV 1 (animals) ARRB
    OBJB 3 (animals.Dog)
      ASGV 7 (id) NATV INT32 1
    OBJE
    OBJB 2 (animals.Cat)
      ASGV 0 (agility_wins) ARRB
        NATV LOCAL_DATE 2025-03-04
        NATV LOCAL_DATE 2026-02-13
      ARRE
      ASGV 7 (id) NATV INT32 2
    OBJE
    OBJB 5 (animals.Fox)
      ASGV 7 (id) NATV INT32 3
    OBJE
    OBJB 3 (animals.Dog)
      ASGV 7 (id) NATV INT32 4
    OBJE
    OBJB 6 (animals.Tiger)
      ASGV 7 (id) NATV INT32 5
    OBJE
  ARRE
OBJE
6

Having looked at the spec, it’s an interesting idea! Though I think it’s a bit unhelpful to describe it as not being a binary protocol. It feels more accurate to describe it as being a binary protocol that’s limited to printable bytes, if I’m understanding correctly. That’s a cool and useful idea that the “not being a binary protocol” messaging kind of obscures, imo

7

Yeah. I kind of understand your point, and this protocol lands somewhat in the middle of the binary and character-based world because it has characteristics from both. Depending on your approach, it could be handled in either way. From current implementations, Python and JavaScript are pure string-based parsers. The Java version kind of tries to lean on the binary approach but not really.

However, what makes it lean more toward the character-based world is how the values are encoded. They stay in human-readable format, like numbers (3233334.2332423) or dates (2025-01-01). In binary, 32-bit numbers would be encoded as their 4-byte binary representation, etc. I did consider some different encodings for numbers. However, because the origin of this is in JavaScript, it makes no sense to try to extract any binary information out of them. And even if it could be done, it would be utterly slower compared to Number.toString() or Number.parseFloat(), which are native C functions.

I’m glad that you brought this up because it is a bit difficult to explain.