
### An Evolution of Data

- **CSV**

A CSV file looks like this :

```csv
ID,LNAME,FMAME,BDATE
1,Jack,Codered,07-02-1997
```

| Advantages    | Disadvantages                               |
| ------------- | ------------------------------------------- |
| Easy to Read  | Type Inferred                               |
| Easy to Parse | Parsing is tricky when data contains commas |

- **DB Tables**

DB tables or Relational DB look like this:

```sql
CREATE TABLE user {
id INTEGER PRIMARY KEY,
lname VARCHAR,
fname VARCHAR,
bday DATE
}
```

| Advantages        | Disadvantages                                    |
| ----------------- | ------------------------------------------------ |
| Data is Typed     | No list ,array,json...                           |
| Focused on Schema | Schema Sharing is hard                           |
| Normalization     | ORM is used to map data in DB to objects in code |

- **JSON**

```json
[
  {
    "name": "Molecule Man",
    "age": 29,
    "secretIdentity": "Dan Jukes",
    "powers": ["Radiation resistance", "Turning tiny", "Radiation blast"]
  }
]
```

| Advantages                      | Disadvantages           |
| ------------------------------- | ----------------------- |
| Easily Sharable                 | Dynamic Schema          |
| Have Arrays                     | Redundancy              |
| High Support and Dynamic Schema | No comments or metadata |

- **Protocol Buffers**

```pb
syntax = "proto3";

import "google/protobuf/timestamp.proto";

message User {
    uint32 id=1;
    string lname = 2;
    string fname = 3;
    google.protobuf.Timestamp bday = 4;
}
```

| Advantages                       | Disadvantages             |
| -------------------------------- | ------------------------- |
| Typed                            | Binary (Human Unreadable) |
| Generate Code and allow comments | Less Support              |
| Schema evolution                 |                           |

### What are Protocol Buffers

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler.Define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

Most common usage of Protocol Buffers are communications and the framework that is most used for that is gRPC (which serialize and deserialize data between the clients and servers).

### Protocol Buffers Basics

- #### Scalar Types

  ##### Numbers

  - int32,int64,sint32,sint64
  - uint32,uint64
  - fixed32,fixed64,sfixed32,sfixed64
  - float,double

  ##### Booleans

  - keyword:bool
  - value: true or false
  - default : false

  ##### String

  - keyword:string
  - value: arbitary length text
  - default : empty string
  - UTF-8 encoded or 7-bit ASCII

  ##### Bytes

  - keyword:bytes
  - value: arbitary length byte sequence
  - default : empty bytes

- #### Tags

  Tags are what makes serialization and deserialization possible.
  Smallest tag is 1 and largest is 536,870,911.
  Reserved tags: 19000 to 19999.
  Smallest tag numbers should be kept for the most populated fields in the schema.

- #### Repeated Fields

  - keyword: repeated <type><name> = <tag>;
  - value: Any number of elements (0 or more)
  - default: empty list

- #### Enums

  - keyword:enum
  - default: first value
  - First tag:0

### Generate Code in Golang

##### Makefile and Generate Go Code

This tool will help us generate go code from proto file.

```makefile
BIN = proto-go-course
PROTO_DIR = proto

ifeq ($(OS), Windows_NT)
	OS = windows
	SHELL := powershell.exe
	.SHELLFLAGS := -NoProfile -Command
	PACKAGE = $(shell Get-Content go.mod -head 1 | Foreach-Object { $$data = $$_ -split " "; "{0}" -f $$data[1]})
else
	UNAME := $(shell uname -s)
	ifeq ($(UNAME),Darwin)
		OS = macos
	else ifeq ($(UNAME),Linux)
		OS = linux
	else
    	$(error OS not supported by this Makefile)
	endif
	PACKAGE = $(shell head -1 go.mod | awk '{print $$2}')
endif

build: 	generate
	go build -o ${BIN} .

generate:
	protoc -I${PROTO_DIR} --go_opt=module=${PACKAGE} --go_out=. ${PROTO_DIR}/*.proto

clean:
	rm ${PROTO_DIR}/*.pb.go
	rm ${BIN}
```

- Use the go mod init command to create a go.mod file to track your code's dependencies.
- Now check if the initialization is working properly with go mod tidy command.
- Create a directory called proto
- Now write a proto file inside proto and use the make generate command to generate go code.

#### Create a Simple Proto Struct in Golang

```proto
syntax = "proto3";

package example.simple;

option go_package = "github.com/codered-jack/proto-go/proto";

message Simple {
  uint32 id = 1;
  bool is_simple = 2;
  string name = 3;
  repeated int32 sample_lists = 4;
}
```

- **main.go**

```go
package main

import (
    "fmt"

	pb "github.com/codered-jack/proto-go/proto"
	"google.golang.org/protobuf/proto"
)

func doSimple() *pb.Simple {
	return &pb.Simple{
		Id:          1,
		Name:        "Jack",
		IsSimple:    true,
		SampleLists: []int32{1, 2, 3, 4, 5, 6},
	}
}

func main() {
	fmt.Println(doSimple())
}
```

#### Reading and Writing to JSON

```go
package main

import (
	"log"

	"google.golang.org/protobuf/encoding/protojson"
	"google.golang.org/protobuf/proto"
)

func toJSON(pb proto.Message) string {
	option := &protojson.MarshalOptions{
		Multiline: true,
	}

	out, err := option.Marshal(pb)

	if err != nil {
		log.Fatalln("Can't convert to JSON", err)
		return ""
	}

	return string(out)
}

func fromJSON(in string, pb proto.Message) {
	option := protojson.UnmarshalOptions{
		DiscardUnknown: true,
	}

	if err := option.Unmarshal([]byte(in), pb); err != nil {
		log.Fatalln("Couldn't unmarshal from JSON", err)
	}
}
```

- **main.go**

```go
func doToJSON(p proto.Message) string {
    jsonString := toJSON(p)
    fmt.Println(jsonString)
    return jsonString
}

func doFromJSON(jsonString string, t reflect.Type) proto.Message {
	message := reflect.New(t).Interface().(proto.Message)
	fromJSON(jsonString, message)
	return message
}

func main() {
    fmt.Println(doFromJSON(`{"id": 1, "unknown": "test"}`, reflect.TypeOf(pb.Simple{})))
}
```

_References_ :

- [Notes From Google Protobuf, Pre-req to gRPC](https://www.udemy.com/course/protocol-buffers/) , Clement Jean
