Install LM Studio and download models

Before we begin, ensure you have the following installed:

• R
• LM Studio, which can be downloaded from the LM Studio website

After you have downloaded and installed LM Studio, open the application. Go to the Discover tab (sidebar), where you can browse and search for models. In this example, we will be using the Phi-3-mini-4k-instruct model, but you can of course experiment with any other model that you prefer – as long as you’ve got the hardware to run it!

Now, select the model from the top bar to load it:

To check that everything is working fine, go to the Chat tab on the sidebar and start a new chat to interact with the Phi-3 model directly. You’ve now got your language model up and running!

Required R Packages

To effectively work with LM Studio, we will need several R packages:

• tidyverse – for data manipulation
• httr – for API interaction
• jsonlite – for JSON parsing

You can install/update them all with one line of code:

# Install necessary packages
install.packages(c("tidyverse", "httr", "jsonlite"))

Let us set up the R script by loading the packages and the data we will be working with:

# Load the packages
library(tidyverse)
library(httr)
library(jsonlite)

top_100_climbs_df <- read_csv("https://raw.githubusercontent.com/martinctc/blog/refs/heads/master/datasets/top_100_climbs.csv")

The top_100_climbs_df dataset contains information on the top 100 cycling climbs in the UK, which I’ve pulled from the Cycling Uphill website, originally put together by Simon Warren. These are 100 rows, and the following columns in the dataset:

• climb_id: row unique identifier for the climb
• climb: name of the climb
• height_gain_m: height gain in meters
• average_gradient: average gradient of the climb
• length_km: total length of the climb in kilometers
• max_gradient: maximum gradient of the climb
• url: URL to the climb’s page on Cycling Uphill

Here is what the dataset looks like when we run dplyr::glimpse():

glimpse(top_100_climbs_df)
## Rows: 100
## Columns: 7
## $ climb_id         <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16…
## $ climb            <chr> "Cheddar Gorge", "Weston Hill", "Crowcombe Combe", "P…
## $ height_gain_m    <dbl> 150, 165, 188, 372, 326, 406, 166, 125, 335, 163, 346…
## $ average_gradient <dbl> 0.05, 0.09, 0.15, 0.12, 0.10, 0.04, 0.11, 0.11, 0.06,…
## $ length_km        <dbl> 3.5, 1.8, 1.2, 4.9, 3.2, 11.0, 1.5, 1.1, 5.4, 1.4, 9.…
## $ max_gradient     <dbl> 0.16, 0.18, 0.25, 0.25, 0.17, 0.12, 0.25, 0.18, 0.12,…
## $ url              <chr> "https://cyclinguphill.com/cheddar-gorge/", "https://…

Our goal here is to use this dataset to generate text descriptions for each of the climbs using the language model. Since this is for text generation, we will do a bit of cleaning up of the dataset, converting gradient values to percentages:

top_100_climbs_df_clean <- top_100_climbs_df %>%
  mutate(
    average_gradient = scales::percent(average_gradient),
    max_gradient = scales::percent(max_gradient)
    )

Setting up the Local Endpoint

Once you have got your model in LM Studio up and running, you can set up a local endpoint to interact with it directly from your R environment.

To do this, go to the Developer tab on the sidebar, and click ‘Start Server (Ctrl + R)’.

Setting up a local endpoint allows you to interact with the language model directly from your R environment. If you leave your default settings unchanged, your endpoints should be as follows:

• GET http://localhost:1234/v1/models
• POST http://localhost:1234/v1/chat/completions
• POST http://localhost:1234/v1/completions
• POST http://localhost:1234/v1/embeddings

In this article, we will be using the chat completions endpoint for summarising / generating text.

Creating a custom function

Below is a code snippet for creating a custom function to communicate with your local LM Studio endpoint:

# Define a function to connect to the local endpoint
send_prompt <- function(system_prompt, user_prompt, endpoint = "http://localhost:1234/v1/chat/completions") {

  # Define the data payload for the local server
  data_payload <- list(
    messages = list(
      list(role = "system", content = system_prompt),
      list(role = "user", content = user_prompt)
    ),
    temperature = 0.7,
    max_tokens = 500,
    top_p = 0.9,
    frequency_penalty = 0.0,
    presence_penalty = 0.0
  )

  # Convert the data to JSON
  json_body <- toJSON(data_payload, auto_unbox = TRUE)
  
  # Define the URL of the local server
  response <- POST(
    endpoint, 
    add_headers(
      "Content-Type" = "application/json"),
    body = json_body,
    encode = "json")

  if (response$status_code == 200) {
    # Parse response and return the content in JSON format
    response_content <- content(response, as = "parsed", type = "application/json")
    response_text <- response_content$choices[[1]]$message$content
    response_text

  } else {
    stop("Error: Unable to connect to the language model")
  }
}

There are a couple of things to note in this function:

1. The send_prompt function takes in three arguments: system_prompt, user_prompt, and endpoint.

• We distinguish between the system and user prompts here, which is typically not necessary for a simple chat completion. However, it is useful for more complex interactions where you want to guide the model with specific prompts. The system prompt is typically used for providing overall guidance, context, tone, and boundaries for the behaviour of the AI, while the user prompt is the actual input that you want the AI to respond to.
• The endpoint is the URL of the local server that we are connecting to. Note that we have used the chat completions endpoint here.

2. The data_payload is a list that contains the messages (prompts) and the parameters that you can adjust to control the output of the language model. These parameters can vary depending on the model you are using - I typically search for the “API documentation” or the “API reference” for the model as a guide. Here are the parameters we are using in this example:

• messages is a list of messages that the language model will use to generate the text. In this case, we have a system message and a user message.
• temperature controls the randomness of the output. A higher temperature will result in more random output.
• max_tokens is the maximum number of tokens that the language model will generate.
• top_p is the nucleus sampling parameter, and an alternative to sampling with temperature. It controls the probability mass that the model considers when generating the next token.
• frequency_penalty and presence_penalty are used to penalize the model for repeating tokens or generating low-frequency tokens.

3. The json_body is the JSON representation of the data_payload list. We need to transform the list into JSON format because this is what is expected by the local server. We do this with jsonlite::toJSON().

4. The response object is the result of sending a POST request to the local server. If the status code of the response is 200, then we return the content of the response. If there is an error, we stop the function and print an error message.

Now that we have our function, let us test it out!

Testing the Function

To ensure your function works as expected, run a simple test:

# Test the generate_text function
test_hill <- top_100_climbs_df_clean %>%
  slice(1) %>% # Select the first row
  jsonlite::toJSON()

send_prompt(
  system_prompt = paste(
    "You are a sports commentator for the Tour de France.",
    "Describe the following climb to the audience in less than 200 words, using the data."
  ),
  user_prompt = test_hill
  )
## [1] "Ladies and gentlemen, hold on to your helmets as we approach the infamous Cheddar Gorge climb – a true testament of resilience for any cyclist tackling this segment! Standing at an imposing height gain of approximately 150 meters over its lengthy stretch spanning just under four kilometers, it demands every last drop from our riders. The average gradient here is pitched at around a challenging 5%, but beware – the climb isn't forgiving with occasional sections that reach up to an extreme 16%! It’s not for the faint-hearted and certainly no place for those looking for respite along this grueling ascent. The Cheddar Gorge will separate contenders from pretenders, all in one breathtakingly scenic setting – a true masterclass of endurance that is sure to make any Tour de France rider's legs scream!"

Not too bad, right?