Generic problem solving

Sometimes you encounter a problem which should be easy to solve theoretically, but you just can't figure it out! In this article I will tackle a problem I've had regarding multiple generics with a pattern called type erasure. I will explain my use case and how I got to the solution.

Aug 19th programming v

You can find the code for this article here.

The goal

During my vacation I got the idea for a new project: a database admin panel! I want to implement it solely using V's built-in ORM , why you might ask, because I can and I think it will be an intersting project. I really want to make a project with the javascript library htmx, I think it's hypermedia driven approach is a perfect fit for V's web framework vweb.

Syntax

I came up with the following syntax that lets users register structs which will be shown in the admin panel, kinda like django's admin panel.

main.v

module main

import admin

struct User {
	id   int    [primary; sql: serial]
	name string
	age  int
}

fn main() {
	admin.register[User]()
    admin.start()
}

That's it! The line admin.register[User]() is where the magic begins. It tells the admin module that we want to show the User struct in the admin panel where we can do all kinds of CRUD operations on the struct.

The line admin.start() starts a vweb application that is the admin panel.

Vweb

Here is where things get to start tricky. For example, how does the frontend tell the backend that it wants to get all rows of the User struct?

Generic routes

Vweb does not accept generic methods, so we can't do something like this:

pub fn (mut app AdminApp) get_struct[T]() vweb.Result {}

When you think about it, it makes sense. How does the V compiler know which types will be passed to get_struct[T]()? That would only be known at runtime and V is a compiled language, so this approach is a no go.

Reflection

But there is a simple solution: reflection! Luckily V has great reflection tools and each type has a unique index, which we can get in our generic register[T]() function.

admin/admin.v

module admin

pub fn register[T]() {
	println('The index of type "${T.name}" is ${T.idx}')
}

Ok, so with that index we can make a dynamic route in vweb and we know which type is requested by the frontend.

admin/web.v

module admin

import v.reflection
import vweb

pub struct AdminApp {
	vweb.Context
}

['/structs/:typs']
pub fn (mut app AdminApp) view(typs string) vweb.Result {
	typ := typs.int()
	type_name := reflection.type_name(typ)
	if type_name.len == 0 {
		// the type does not exist!
		return app.not_found()
	}

	return app.text('Requested type (idx=${typ}, name="${type_name}")')
}

pub fn start() {
	vweb.run(&AdminApp{}, 8000)
}

If you run the code you will see the index of our User struct logged and when we visit the url we can indeed see our User struct!

Screenshot of browser; your type index might be different

Adding functionality

Lets add some more functionality to the admin module. I want to be able to press a button and a new record of that type is added to the database.

admin/templates/view.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Create struct</title>
</head>
<body>
<p>Requested type (idx=@typ, name="@type_name")</p>
<form action="/create/@typ" method="POST">
    <button type="submit">Create row of @type_name</button>
</form>
</body>
</html>

Updated admin/web.v

['/structs/:typs']
pub fn (mut app AdminApp) view(typs string) vweb.Result {
	typ := typs.int()
	type_name := reflection.type_name(typ)
	if type_name.len == 0 {
		// the type does not exist!
		return app.not_found()
	}

	return $vweb.html()
}

['/create/:typs'; post]
pub fn (mut app AdminApp) create(typs string) vweb.Result {
	typ := typs.int()

	create_row(typ)

	return app.text('Created row!')
}

pub fn start() {
	vweb.run(&AdminApp{}, 8000)
}

The Problem

This is where the problem occurred, maybe you have already spotted it. How are we going to implement create_row? Because we can't create an instance of a type from its index at runtime.

I've tried all sorts of things like getting the full type and trying create an object from that type. Somehow this code compiled:

module main

import v.reflection

struct User {
	age int
}

fn create_row(idx int) reflection.Type {
	typ := reflection.get_type(idx) or { return reflection.Type{} }
	return typ
}

fn main() {
	user_idx := typeof[User]().idx
	user := create_row(user_idx){
		age: 5
	}
	println(user)
}

But it didn't actually create an object of type User.

It is all C

I already mentioned it earlier, but when I encounterd this problem I forgot that I can't do this at runtime. V is transpiled to C and then converted to an executable. Looking at the problem from this perspective I knew that this approach wasn't going to work.

How would this work in C? It doesn't. You would need to create types depending on the input at runtime. See the following example:

#include <stdio.h>

typedef struct User {
    int age;
} User;

int user_idx = 96;

void create_row(int idx) {
    // V would need to generate if statements for every type 
    // in the program and V itself
    if (idx == user_idx) {
        printf("Creating instance of User\n");        
        User obj;
        // What now???
    } else {
        printf("Unkown type!\n");
    }
}

int main() {
    create_row(user_idx);
}

V would need to generate if statements for every type in the program and type V itself uses. And even if this would be possible how are we going to continue this function? There are more approaches to this problem, but they will all fail. I couldn't wrap my head around it.

The solution

About a week later when I was walking it suddenly occured to me that I already solved this problem before! Specifically, when I implemented the Controllers feature in vweb.

Vweb: add controllers by Casper64 · Pull Request #17840 · vlang/v

Vweb add ability to use controller so you are able to have one struct per "/". E.g. a struct Admin for urls starting with "/admin" and a struct Foo for urls starting with &quot...

https://github.com

The function controller[T]() returns an object that has a function which encapsulates the generic type. Big words, what does that mean? Let's look at a code example:

module main

struct User {
pub mut:
	name string
	age  int
}

struct Employee {
pub mut:
	id   int
	name string
}

struct Model {
pub:
	name       string
	idx        int
	create_row fn ()  [required]
}

fn create_model[T]() Model {
	return Model{
		name: T.name
		idx: T.idx
		create_row: fn [T]() {
			mut obj := T{}
			println('We have an object of "${T.name}": ${obj}')
		}
	}
}

fn main() {
	// stores all the models
	mut models := map[string]Model{}

	models['User'] = create_model[User]()
	models['Employee'] = create_model[Employee]()

	models['User'].create_row()
	models['Employee'].create_row()
}

When you run this code you will see the following output:

We have an object of "User": User{
    name: ''
    age: 0
}
We have an object of "Employee": Employee{
    id: 0
    name: ''
}

This patterns is called type erasure, because we have essentially erased the type of User and Employee from the Model struct.

The line create_row: fn [T]() { tells V that we want to pass the type of T to the anonymous function.

Applying type erasure

When you word it in other terms the type erasure pattern is basically a generic constructor. The only restriction with this pattern is that we have to define all of our functionality in the scope of where we create an instance of Model. We can ofcourse use the generic type then again in functions in the global scope.

Calling other generic functions

We can pass the generic type to other generic functions in create and it will work just fine.

fn create_model[T]() Model {
	return Model{
		name: T.name
		idx: T.idx
		create_row: fn [T]() {
			mut obj := T{}
			println('We have an object of "${T.name}": ${obj}')
			other_generic_func[T]()
		}
	}
}

fn other_generic_func[T]() {
	println('Called other generic function with the type "${T.name}"')
}

Modifying our code

With some adjustments we can modify our register[T]() function to implement type erasure.

Updated admin/admin.v

module admin

const (
	structs = map[int]Model{}
)

struct Model {
	typ    int
	name   string
	create fn ()  [required]
}

pub fn create_row(idx int) {
	model := admin.structs[idx] or { return }
	model.create()
}

pub fn register[T]() {
	println('The index of type "${T.name}" is ${T.idx}')

	// const hack
	mut mutable_structs := unsafe { admin.structs }

	mutable_structs[T.idx] = Model{
		typ: T.idx
		name: T.name
		create: fn [T]() {
			obj := T{}
			println('Created instance of "${T.name}": ${obj}')
			// do stuff with the database
		}
	}
}

The line mut mutable_structs := unsafe { structs } is referred to as the "const hack" in V. It is a hack where you make a global constant mutable, the constant has to be a pointer. Overall bad practice, but I don't want to compile with -use-globals everytime. Also the constant is limited to the admin module only, so it should be safe if we handle it correctly.

When we run the code and click on the "create row" button we will see that the create function is called with our User struct!

The index of type "User" is 96
[Vweb] Running app on http://localhost:8000/
[Vweb] We have 23 workers
Created instance of "User": User{
    id: 0
    name: ''
    age: 0
}

Conclusion

With this new functionality I can begin to implement the database backend functions that will create, read, update and delete rows. Unfortunately V's ORM is not generic, but it should be usable with only using reflection (new article spoiler alert!).

Sometimes you have to think simple.

You can find the code for this article here.