Forked from schweigert/Embedding GoLang into a Ruby application.md
Created
September 26, 2023 19:15
-
-
Save metacritical/e5d624715363ffe0370dbf247594e6b5 to your computer and use it in GitHub Desktop.
Revisions
-
Marlon Henry Schweigert revised this gist
Jan 30, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -37,7 +37,7 @@ You need to tell the Go compiler which functions will be public in the library b ```golang import "C" //export my_add func my_add(a, b C.int) C.int { return a + b } -
Marlon Henry Schweigert revised this gist
Jan 30, 2018 . 1 changed file with 0 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -30,8 +30,6 @@ Remember that this is the Achilles' heel of Go Lang. Try to generate a small interface, where only few data is carried on. In general, data type translation is extremely slow when converting from Go to C types! ## Function export signature You need to tell the Go compiler which functions will be public in the library by adding a comment above your function: -
Marlon Henry Schweigert revised this gist
Jan 30, 2018 . 1 changed file with 25 additions and 32 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,42 +1,40 @@  I am passionate about Ruby, but its execution time compared to other languages is extremely high, especially when we want to use more complex algorithms. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. Some algorithms such as ´n-body´ and ´fannkuch-redux´ [can be up to 30 times slower in Ruby than Go](https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=go). This is one of the reasons I was interested in embedding Go code in a Ruby environment. For those who do not know how shared libraries operate, they work in a similar way as DLLs in Windows. However, they have a native code with a direct interface to the C compiler. > **Note** > Windows uses the DLL system, and in this case, this does not necessarily have to be in native code. > > One example is DLLs written in C#, which runs on a virtual machine. Because I do not use windows, I ended up not testing if it is possible to perform these steps on it. > > You need Golang 1.5 or higher and Ruby 1.8.7 or higher. # 1. Create your Go library! It would be like any library, but it must have some peculiar characteristics. ## Use C language types For signing external functions to your library, you will need to use the C language typing. For the rest of the code, you can convert to the standard Go typing. To do this include library: ```golang import "C" ``` Remember that this is the Achilles' heel of Go Lang.  Try to generate a small interface, where only few data is carried on. In general, data type translation is extremely slow when converting from Go to C types! [NÃO ENTENDI] If you create small functions in Golang and use it in Ruby, you'll probably lose time! ## Function export signature You need to tell the Go compiler which functions will be public in the library by adding a comment above your function: ```golang import "C" @@ -55,34 +53,30 @@ func main() { Remember that the exported functions must be in your main package. > **Note** > I know that the standard of nomenclatures in Go is different from what we found in this example, but I've created the habit of writing interfaces using the Ruby style. For this reason, the names of the functions are written using `snake_case` and not `camelCase`. ## Compile Just type: ```sh go build -o my_lib.so -buildmode=c-shared my_file.go ``` # Writing the interface with the shared library in Ruby This is the fastest way to embed Go code into a Ruby environment, however, remember that in this way the call time of the go interface will not be so fast. ## Preparing the environment Add to your `Gemfile` the following gem: ```ruby gem 'ffi' ``` Or of course, manually install it: `gem install ffi` ## Create a module for your library Just create a module and write your function signatures inside the Ruby package. This is like a `.h` file: ```ruby require 'ffi' @@ -98,8 +92,7 @@ puts Foo.my_add(2, 2) # => 4 ``` # What I noticed by running native Go code in the Ruby environment? Integrating Go and Ruby is fast and efficient for certain situations. Running the native code in Go brings several advantages, bringing you various graphical, database and efficient data structures libraries already implemented in Go and the efficiency of Go routines. Remember that this applies when we use little data translations between Ruby and Go. This will only be advantageous if you have a huge real gain like the algorithms I quoted above, ´n-body´ and ´fannkuch-redux´, offsetting this loss by calling a function through the Go C Library. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 0 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -71,9 +71,6 @@ This is the fastest way to embed Golang into Ruby environment, however, we remem ## Preparing the environment Add to your gemfile the following line: ```ruby -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,3 +1,5 @@  I am passionate about ruby, but its execution time compared to other languages is extremely high, especially when we want to use algorithms of greater complexity. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. Some algorithms such as ´n-body´ and ´fannkuch-redux´ can be up to 30 times slower in ruby [Ruby vs Go - Benchmark Game](https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=go). This is one of the reasons I was interested in use embedded GoLang code in a ruby environment. For those who do not know the operation of the shared libraries, they work in a similar way as the dll files in windows, -
Marlon Henry Schweigert renamed this gist
Jan 29, 2018 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 3 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -101,4 +101,6 @@ puts Foo.my_add(2, 2) # What I noticed with the native GoLang code in the ruby environment? How to integrate Golang and Ruby is fast (in time to integrate) and efficient for certain situations. Running the native code in GoLang brings several advantages, bringing you various graphical, database and efficient data structures libraries already implemented in GoLang and the efficiency of Go rotinies. Remember that this applies when we use little data transition between Ruby and GoLang. This will only be advantageous if you have a huge real gain like the algorithms I quoted at the beginning of the post, offsetting this loss by calling a function through the C Library of GoLang. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -24,7 +24,7 @@ import "C" ``` Remember that this is the Achilles' heel of Go Lang.  Try to generate a small interface, where only a few data is carried here. In general, data translation is extremely slow! -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ I am passionate about ruby, but its execution time compared to other languages is extremely high, especially when we want to use algorithms of greater complexity. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. Some algorithms such as ´n-body´ and ´fannkuch-redux´ can be up to 30 times slower in ruby [Ruby vs Go - Benchmark Game](https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=go). This is one of the reasons I was interested in use embedded GoLang code in a ruby environment. For those who do not know the operation of the shared libraries, they work in a similar way as the dll files in windows, however it is a native code with a direct interface to the C compiler. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 5 additions and 7 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -27,7 +27,9 @@ Remember that this is the Achilles' heel of Go Lang.  Try to generate a small interface, where only a few data is carried here. In general, data translation is extremely slow! If you create small functions in GoLang and use it in Ruby, you'll probably lose time! ## Function export signature @@ -81,7 +83,7 @@ Or of course, install manually: `gem install ffi` ## Create a module for your library Just create a module and write your function signatures inside the ruby package. This is like a `.h` file. ```ruby require 'ffi' @@ -99,8 +101,4 @@ puts Foo.my_add(2, 2) # What I noticed with the native GoLang code in the ruby environment? How to integrate Golang and Ruby is fast (in time to integrate) and efficient for certain situations. Running the native code in GoLang brings several advantages, bringing you various graphical, database and efficient data structures libraries already implemented in GoLang and the efficiency of Go rotinies. Remember that this applies when we use little data transition between Ruby and GoLang. This will only be advantageous if you have a huge real gain like the algorithms I quoted at the beginning of the post, offsetting this loss by calling a function through the FFI interface. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -24,7 +24,7 @@ import "C" ``` Remember that this is the Achilles' heel of Go Lang.  Try to generate a small interface, where only a few data is carried here. In general, string translation is extremely slow. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ I am passionate about ruby, but its execution time compared to other languages is extremely high, especially when we want to use algorithms of greater complexity. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. Some algorithms such as ´n-body´ and ´fannkuch-redux´ can be up to 30 times slower in ruby (Ruby vs Go - Benchmark Game)[https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=go]. This is one of the reasons I was interested in use embedded GoLang code in a ruby environment. For those who do not know the operation of the shared libraries, they work in a similar way as the dll files in windows, however it is a native code with a direct interface to the C compiler. -
Marlon Henry Schweigert revised this gist
Jan 29, 2018 . 2 changed files with 106 additions and 110 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,110 +0,0 @@ This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,106 @@ I am passionate about ruby, but its execution time compared to other languages is extremely high, especially when we want to use algorithms algorithms of greater complexity. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. Some algorithms such as ´n-body´ and ´fannkuch-redux´ can be up to 30 times slower in ruby (Ruby vs Go - Benchmark Game)[https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=yarv&lang2=go]. This is one of the reasons I was interested in use embedded GoLang code in a ruby environment. For those who do not know the operation of the shared libraries, they work in a similar way as the dll files in windows, however it is a native code with a direct interface to the C compiler. Note: - Windows uses the DLL system, and in this case, this does not necessarily have to be in native code. One example is DLLs with C #, which runs on a virtual machine. Because I do not use windows, I ended up not testing if it is possible to perform these steps on this operating system. - You need GoLang 1.5 or higher and ruby 1.8.7 or higher. # 1. Create your GoLang library! It would be like any library, but it must have some peculiar characteristics. ## Use C language types For signing external functions to your library, you will need to use the C language typing. For the rest of the code you can convert to the standard GoLang typing. To do this include library: ```golang import "C" ``` Remember that this is the Achilles' heel of Go Lang.  Try to generate a small interface, where only a few data is carried here. In general, string translation is extremely slow. ## Function export signature You need to tell the golang compiler which functions will be public in the library. For this you should create a comment above your function. ```golang import "C" //export MyAdd func my_add(a, b C.int) C.int { return a + b } func main() { // This is necessary for the compiler. // You can add something that will be executed when engaging your library to the interpreter. } ``` Remember that the exported functions must be in your main package. Note: I know that the standard of nomenclatures in Golang are different from what we found in this example, but I've created the habit of writing interfaces with the same ruby pattern. For this reason the names of the functions are written with `snake_case` and not in `camelCase`. ## Compile Just type: ```sh go build -o my_lib.so -buildmode=c-shared my_file.go ``` # Writing the interface with the shared library in ruby This is the fastest way to embed Golang into Ruby environment, however, we remember that in this way the call time of the go interface will not be so fast. ## Preparing the environment You need to be in an environment `**ix`. That is, linux or MacOS. We will need these two systems because the gem we will use only guarantees its operation in these two environments. Add to your gemfile the following line: ```ruby gem 'ffi' ``` Or of course, install manually: `gem install ffi` ## Create a module for your library Just create a module and write your function signatures inside the ruby package. This will act as your library `.h` file. ```ruby require 'ffi' module Foo extend FFI::Library ffi_lib './my_lib.so' attach_function :my_add, [:int, :int], :int end puts Foo.my_add(2, 2) # => 4 ``` # What I noticed with the native GoLang code in the ruby environment? How to integrate Golang and Ruby is fast (in time to integrate) and efficient for certain situations. Running the native code in GoLang brings several advantages, bringing you various graphical, database and efficient data structures libraries already implemented in GoLang and the efficiency of Go rotinies. Remember that this applies when we use little data transition between Ruby and GoLang. This will only be advantageous if you have a huge real gain like the algorithms I quoted at the beginning of the post, offsetting this loss by calling a function through the FFI interface. If it is an algorithm that has little time gain, we recommend using the same strategy, but with pure C or C ++. If this is a case where the execution gain is a few seconds, you will surely lose performance. -
Marlon Henry Schweigert revised this gist
Jan 23, 2018 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -15,6 +15,10 @@ Because I do not use windows, I ended up not testing if it is possible to perfor It would be like any library, but it must have some peculiar characteristics. ## Preparing the environment You need GoLang 1.5 or higher. ## Use C language types For signing external functions to your library, you will need to use the C language typing. -
Marlon Henry Schweigert revised this gist
Jan 23, 2018 . No changes.There are no files selected for viewing
-
Marlon Henry Schweigert revised this gist
Jan 23, 2018 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -85,7 +85,7 @@ require 'ffi' module Foo extend FFI::Library ffi_lib './my_lib.so' attach_function :my_add, [:int, :int], :int end -
Marlon Henry Schweigert created this gist
Jan 23, 2018 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,106 @@ I am passionate about ruby, but to date it is complicated to implement more complex data structures. In general, data structures in interpreted languages become incredibly slow compared to compiled languages. This is one of the reasons I was interested in using GoLang code snippets within the ruby environment. For this, I studied how to export compiled golang codes to shared libraries. For those who do not know the operation of the shared libraries, they work in a similar way as the dll files in windows, however it is a native code with a direct interface to the C compiler. Note: Windows uses the DLL system, and in this case, this does not necessarily have to be in native code. One example is DLLs with C #, which runs on a virtual machine. Because I do not use windows, I ended up not testing if it is possible to perform these steps on this operating system. # 1. Create your GoLang library! It would be like any library, but it must have some peculiar characteristics. ## Use C language types For signing external functions to your library, you will need to use the C language typing. For the rest of the code you can convert to the standard GoLang typing. To do this include library: ```golang import "C" ``` ## Function export signature You need to tell the golang compiler which functions will be public in the library. For this you should create a comment above your function. ```golang import "C" //export MyAdd func my_add(a, b C.int) C.int { return a + b } func main() { // This is necessary for the compiler. // You can add something that will be executed when engaging your library to the interpreter. } ``` Remember that the exported functions must be in your main package. Note: I know that the Golang standard for function names is not this, but it is the default for functions in C and Ruby. For this reason, I got used to writing the functions that will be exported to the same pattern. ## Compile Just type: ```sh go build -o my_lib.so -buildmode=c-shared my_file.go ``` # Writing the interface with the shared library in ruby There are more efficient ways to write native Ruby code, but this is the most readable and efficient way I've found. ## Preparing the environment You need to be in an environment `**ix`. That is, linux or MacOS. We will need these two systems because the gem we will use only guarantees its operation in these two environments. Add to your gemfile the following line: ```ruby gem 'ffi' ``` Or of course, install manually: `gem install ffi` ## Create a module for your library Just create a module and write your function signatures inside the ruby package. This will act as your library `.h` file. ```ruby require 'ffi' module Foo extend FFI::Library ffi_lib FFI::Library::LIBC attach_function :my_add, [:int, :int], :int end puts Foo.my_add(2, 2) # => 4 ``` # What I noticed using native code in GoLang? Running the native code is very fast, however, the interface call is not so fast. It's great to use this to solve complex computational problems which involve dynamic programming, graphs, ad hoc and others. Some issues I solved in go lang took a thousandths of seconds compared to seconds in ruby. Another point is the care when writing the exported functions. The C functions have no garbage collector! Parallelism in GoLang is MUCH BETTER than in Ruby. Think about it carefully and lovingly.