Crate wui 

READ THIS!

Versions below 0.2.0 are no longer supported, they are from the Cpp build.

You will not be able to download the source for this crate at this time! I am busy rewriting it and will make it available here once I know I want to. The pre compiled .so/.rlib files should be available for download though on https://static.zweieuro.at/.

The library is MUCH smaller. I am not packaging cef itself anymore, if you want to use this library get the apropriate distribution (minimal) from https://cef-builds.spotifycdn.com/index.html. You can click the dependency for cef of the crate to get the version. The SECOND number is what you want (e.g. 141.0.11) The first number is the version for the cef,cef-dll-sys and export-cef-dir crates.

NOTE: This is a very large readme, originally there was a webpage for documentation, but for a lot of reasons I axed it and built this instead. The entire first section is about documentation and about the project itself. It can be safely skipped if you do not mind the internals. Example videos are also seen in that section.

Documentation - Former docusaurus page

TODO: Port this over

The HTML side:

WUI can be used to display almost any webpage that is legal/possible to render by a chromium based web browser, but it was designed to handle “special” pages. The webpages running have a modified global object that exposes some additional functions from inside WUI.

This object can be used to communication bi-directionally between the front and backend of the application. Meaning that data can be pushed to/from the front end through defined channels.

To make this MUCH easier please use this library https://jsr.io/@wui/web-user-interface-lib/. It exposes useful functions like sendEvent, registerEventListener or unregisterEventListener.

It also features a replayTool that can replay log files that WUI creates independently of the backend. great for finding UI bugs without running the backend at all.

Data format

The nature of this system dictates that the data that is forwarded is JSON compatible. Function pointers cannot meaningfully cross the process barrier. Other than that there are no real restrictions. CEF3 notes that after 300kb the data is relayed via a sperately shared memory region but even within testing I could not run into any meaningful performance problems, but this might be relevant for smaller machines.

Though if you are planning to parse a 300kb json I think you should seriously rethink your design.

WUI - Rust Web User Interface Library

A Rust implementation of a Web User Interface library using CEF (Chromium Embedded Framework). You can run any browser-compatile url inside a headless process. The tab is fully interactable (programmatically) via keyboard and mouse input events. The actual “image” that the browser renders is given in BGRA (because that is what it renders as) data and can then be processed anywhere. This makes it ideal to use as an overlay-driver for in-game Uis or menus! This also opens the door to several accessibility, usability, testability and quality assurance tools which are available for the web but not really for native game Uis.

It is highly recommended to use this library with the npm library that exposes its internal features more cleanly (double check the version number! I don’t know when i will get around to fixing the compatibility with it.)

This library provides a native interface for embedding web-based UIs in Rust applications with bidirectional communication between the native code and web content via the internal message router.

Originally I wrote this in C++ but cmake and vcpkg are really not my idea of a good or stable build environment. Therefore I rewrote it in rust, giving it a cleaner concurrency and i.g. IPC implementation than before.

In general this rewrite was a great success I think.

Features

• Multi-process architecture with browser and render processes

• Offscreen rendering with pixel buffer access

• Bidirectional DOM ↔ Native communication via JSON

• Mouse and keyboard input handling

• TODO: Cookie management

Notes on architecture

So how does this broadly work? In essence, from how I understand CEF this is roughly equal:

A “Browser Tab” is in CEF terms a CefBrowser/CefBrowserHost running a single CefClient. The CefClient is responsible for all the stuff “inside” the tab, audio, file handling, drag and drop, context click, etc. The CefBrowser is responsible for interacting with the tab from the “outside”: click/Keyboard events, download, find in page, dev tools etc.

This essentially means that there is a 1:1 relation between client and browser. When I read this I immediately think “oh, so clients are tabs and the browser is the window?”. No, at least not “fully”. There is an additional layer “around” the browser that is then the window handler. Also there is no API to reload or change the loaded client inside any one CerBrowser.

Why is that an issue? Because of refcounting. Cef has a rather strict refcounting system to ensure there is no dangling memory. This is important to any implementation of these systems because you cannot safe CefClient or CefBrowser handles (ref counted pointers) in a structure that facilitates the functionality of the other, otherwise you will end up with a kind-of circular dependency where deleting one cannot delete the other implicitly because of an active ref to self. I’ve found this out pretty much by experimenting with the API.

How to resolve this? Actually easy in general, but it makes the code more fractured. Save the information in an external third struct. That is pretty much the main purpose of the “TabManager”. It holds a struct for the actual tab needed AND it’s browser. All click/keyboard (etc.) events go through the TabManager to the CefBrowser. All rendering/tab internal functionality goes through the TabManger to the CefClient (Tab) directly.

WUI Types

Tab

A tab wraps around a single cef browser. It represents a loaded webpage. All tabs are first created, but due to the multiprocess system it will take a bit for it to actually be created in memory. There are condvars and mutex systems in place to wait for specific states for the tab. They all essentiall go through this lifecycle:

• create_tab() -> Created
• LifeSpanHandler, RenderHandler and LoadHandler all get initialized and get their internal browser object set.
• The load handler is last, initial tabs always load about:blank. As soon as the onLoadFinish event is fired we switch to -> Navigatable
• The tab will stay in this state indefinitely until
• close_tab -> will STAY in Navigatable (NOTE: This should probably be changed to “closing” or something)
• When everything is cleared and deleted, the destructor changes the state to be -> Destroyed

Once destroyed, there is no way to revive a single tab. All id’s are single use.

CEF datatypes

Generally anything we do with CEF is refcounted internally. therefore no wrapping Arc is needed.

The interface is designed to not expose CEF’s types directly and make them easier to use.

CefTab

The tab needs to know its current size. Each draw call queries the “CefClient” for the current view-rectangle. So functionaly we have no choice but to forward this information to the tab and hold it there is a copy. Events for painting and loading also originate in this client so we need to hold function pointers to the outside to fire them when applicable.

CefBrowser

Other than “closing” the entire tab down, the browser is mostly only used to interact with the input method directly.

Context, Request, Queryies, Events and lifetime

FAT NOTE ON CONTEXTS: There are multiple contexts depending on what you are trying to do. Generally the context objects between Create, Release and “V8 handling” are NOT the same object. At least when you compare the pointer adresses you will always get VERY different results. This means we only really care about 1 context, the context in which we execute our JS functions, which is the V8 one. This also mans we can’t intelligently handle Create and Release. We can only really tell the renderer that those requests are no longer valid (the listeners are gone) but that’s about it.

Terminology:

• Requests are everything between JS and the renderer process. They originate in the V8 handler via a bound function
• Queries are end between the renderer and browser process. They are prett much a wrapper around the IPC system that CEF already implements.
  • No functions can directly pass this boundary, that’s where most of the complexity comes from
• Events: Some events are hard to track when it comes to control. The original implementation had a few events:
  • Renderer origin:
    • Create: This usually happens only once. Usually after Webkit initializes (OnWebkitIntitialized). Doing this globally behind a lock is fine as well.
    • OnContextCreated: “Context” refers to the Javascript V8 context. In other words: The tab is alive, loaded, and ready to process JS/HTML. Here we attach our side of the infrastructure to the V8 in order to receive info from JS
    • OnContextReleased: The tab is dropped, or reloaded, or a different tab is loaded. This functionally means ALL standing requests on the JS side are invalidated.
    • OnProcessMessageReceived: Received some IPC, mainly from the browser
  • Browser origin:
    • Handler logic: On the Cpp wrapper for the router there is a lot of handler logic as it tries to wrap a more general case. We don’t haev that here.
    • OnBeforeClose: Called when the tab is closed. Needed to drop all resources on the renderer side.
    • OnBeforeBrowse: Originally this was before a tab was loaded.
    • OnRenderProcessTerminated: This means the process crashed or has to restart, hard to say when this happens but its smart to have. Functionally it means that the browser needs to re-attach all the callbacks that might exist
    • CancelPending: In the original requests could “pend” but this functionality does not exist in this version.

Lifetime:

Request: A request only lives once, it is equivalent to a single function call on the JS side. It can be fulfilled multiple times. The only callback that is stored for more than 1 resolve is the listener logic and the “onData” callback. Though the entire request is cached. Usually at the end of handling the round trip of JS -> Renderer -> Browser -> Renderer the request is “fulfilled” and thereby deleted from the map. This does not happen for “Listening” requests since they can be fulfilled many times. This means we have to have some special logic that keeps them alive when processed.

Queries: Short lived. They represent a single IPC. As soon as its on the other side it gets handled and removed. They should not be stored or copied somewhere as the IDs of the JS request they link to might go invalid really fast.

Events: Events almost instantly translate to functionality or an IPC that is then handeld immediately.

Lifetime of handlers:

Rust: On the rust end handlers are registered through the browser, sent to the Renderer, where they are stoerd. Should that event occure, then the Renderer will generate the right IPC to tell the browser to forward the data to the function. The callback is processed, and then, depending on reject or resolve, the info is sent back to the renderer, which calls the functions in JS.

Registered listeners have indefinite runtime (assuming they are not removed manually or the renderer crashes). Even if the tab reloads and drops it’s context, the function is still inside the renderer proecss. Any future event received from JS is still processed as expected.

Javascript: This is a bit more complex because of the potential context changes. Sending events is unaffected, the singular nature of them makes them simplistic to process. Callbacks on the other hand have to be dropped since there is no way for the renderer to know what function to call after a context switch. This functionally means that all callbacks need to be re-attached (from the JS side, we cannot force it from the rust side) after a new context was created. This also can’t be done automatically since we don’t know where to attach that callback to.

The end result is not as bad as that sounds though. Usually when using hooks or fixed functions, they run when the site is loaded. The only thing needed ist to tell the browser that there is not registered listener if it wants to send an event during that time.

OnContextReleased: Drop Javascript listener callbacks, inform browser that those callbacks are no longer valid and are dropped (can piggy back off of “unlisten” functionality) OnRendererProcessTerminated: Re-attach rust callbacks (can be done automatically, can stay on the browser side) OnBeforeClose: Drop all resources on either side. (termination)

Using in C

C API

A lot of this project reaches through to cef_dll_sys and the cef crate. Those creates define all of the enums that CEF is used to and is expecting to receive. When this project sues cbindgen to generate bindings, those pass-through enums break. Generally cbindgen exports everything with #[repr(C)] which the two cef crates do not derive.

In order to get this into a working state I had to mirror a lot of the enums that cef uses and mark them as C compatible. Not all of the C wrapper functions have the ‘same’ signature as the rust functions because of how struct and strings work in C.

Note: I had massive difficulties getting CEF to initialize and it was due to the library ordering, if you are experiencing problems make sure to look for this.

usage

Warning: The C bindings are not complete or very well tested. I simply didn’t get around to it yet.

There are essentially multiple libraries at play here but for a program that uses WUI start it needs to find libwui.so itself. The resource path is used by the initializer to find the rest of the CEF dependencies.

To test if the library can be called at all from C: A simple C file inside target/debug/c_test.c after compilation:

#include <pthread.h>
#include <stdio.h>

#include "bindings.h"

int main(int argc, const char *argv[]) {

  struct WuiConfigC config = {

      .devtools_mode = Enabled,
      .remote_debugging_port = 8088,
      .locale = "en-US",
      .cache_directory_path = "./cache",
      .wui_resources_path = "./",
      .max_tab_frame_rate = 30,
      .no_sandbox = true};

  const int res = wui_init_c(config);

  if (res == -1) {
    printf("Could not init WUI\n");
    exit(-1);
  }

  const int id = create_tab_c();

  if (id == -1) {
    printf("Could not create tab\n");
    exit(-1);
  }

  const TabId t1 = {id};

  wait_for_tab_state_c(t1, Navigatable);

  printf("Loading url\n");
  load_url_tab_c(t1, "https://example.com");

  printf("exit  main thread\n");
  // let it run until its shot down
  pthread_exit(NULL);

  return (0);
}

• All wui functions should be visible.
• completion and the struct should work.
• everything should be callable.

compiling:

# compile and link with the library
# please note the library order! it caused me a lot of headache
gcc -g -Wall -std=c99 -I. c_test.c -L. -lwui -lpthread -lcef -Wl,-rpath='${ORIGIN}' -o test

# if you compile with rpath then the following `LD_LIBRARY_PATH` is not needed.
LD_LIBRARY_PATH=. ./test

Re-exports

pub use crate::tab::*;

pub use crate::tabs::*;

Modules

c_api

cef_mirror

ipc

keyboard

Keyboard input handling

mouse

tabmanager

tabs

utils

wui_api

Main library control

wui_binding_api

For controlling individual tabs

wui_cookie_api

wui_error

Error types for the RUI library

wui_input_api

API for controling individual input sources and events, keyboard and mouse

wui_logging_api

Main API for logging control feature

wui_tab_api

For controlling individual tabs

Macros

check_tab_exists

Convenience macro for checking if a tab exists

check_tab_has_browser

Convenience macro for checking if a tab has a browser instance

Structs

WuiState

WuiVersion

Version information for the RUI library

Constants

GIT_BRANCH

GIT_COMMIT

GIT_TAG

VERSION

Functions

check_initialized

is_initialized

wui_get_state

Get the current verision blocks

wui_get_version