LinkOrgs Package Tutorial

2025-01-15

LinkOrgs Tutorial

Organizations can appear in different datasets under slightly different names, or with substantially different aliases. Manually linking records can be time-consuming, especially when the datasets become large. LinkOrgs is an R package designed to automate much of this process. It leverages both string-based methods (e.g., fuzzy string matching), and machine-learning or network-based approaches trained on half a billion open-collaborated records from LinkedIn.

In this tutorial, we will show you how to:

Throughout, we will provide practical examples you can run immediately. For a broader explanation and academic references behind these methods, please see the GitHub repository and associated paper.

Installation

You can install LinkOrgs directly from the GitHub repository as follows:

# install.packages("remotes") # if you haven't installed 'remotes' previously
# remotes::install_github("cjerzak/LinkOrgs-software/LinkOrgs")

This will install the package along with its R-based dependencies.

However, if you plan on using the machine learning (ML) linkage methods (algorithm = "ml" in LinkOrgs()), you also need Python packages such as JAX, Optax, equinox, and jmp. We bundle everything in a conda environment for convenience, so you should run BuildBackend() (described below) first.

Building the Conda Backend (Optional, for ML-Based Matching)

If you plan to rely only on fuzzy or network-based matching (as opposed to embedding-based ML matching), you can skip this step. But if you want to get the best performance from the advanced ML-based model, follow these steps to create a conda environment that has all the required Python dependencies:

library(LinkOrgs)
#> Warning: replacing previous import 'Rfast::transpose' by
#> 'data.table::transpose' when loading 'LinkOrgs'

# This function calls reticulate::conda_create() under the hood to create a new conda env
#LinkOrgs::BuildBackend(conda_env = "LinkOrgsEnv",  # or choose custom name
             #conda = "auto", tryMetal = TRUE)

Tip: If you see any errors about “Conda environment not found,” check that you have conda properly installed and that reticulate can locate it.

Core Matching via LinkOrgs()

The main workhorse function for linking two datasets by organization names is LinkOrgs(). It provides multiple methods under the hood:

Minimal Example (Fuzzy Matching)

If you just want to do a quick fuzzy match on two small data frames, run:

# Let's create two synthetic datasets:
x_orgnames <- c("apple","oracle","enron inc.","mcdonalds corporation")
y_orgnames <- c("apple corp","oracle inc","enron","mcdonalds co")

x <- data.frame("orgnames_x" = x_orgnames)
y <- data.frame("orgnames_y" = y_orgnames)

# A straightforward fuzzy match with default settings:
linkedOrgs_fuzzy <- LinkOrgs(x = x,
                             y = y,
                             by.x = "orgnames_x",
                             by.y = "orgnames_y",
                             # MaxDist is a maximum allowed fuzzy distance
                             MaxDist = 0.5,
                             DistanceMeasure = "jaccard",
                             algorithm = "fuzzy")
#> [1] "[2025-01-15 23:01:31] Pre-processing strings..."
#> [1] "[2025-01-15 23:01:31] Searching for matches in the raw name space..."
#> [1] "[2025-01-15 23:01:31] Calibrating via AveMatchNumberPerAlias..."
#> Warning in e$fun(obj, substitute(ex), parent.frame(), e$data): already
#> exporting variable(s): split_list_x, DistanceMeasure, qgram, nCores,
#> ReturnProgress, x_tri_index, y_tri_index, MaxDist, x, by.x, y, by.y
#> [1] "[2025-01-15 23:01:32] Calibrated accept match dist threshold: 0.920000"
#> Warning in e$fun(obj, substitute(ex), parent.frame(), e$data): already
#> exporting variable(s): split_list_x, DistanceMeasure, qgram, nCores,
#> ReturnProgress, x_tri_index, y_tri_index, MaxDist, x, by.x, y, by.y
#> [1] "[2025-01-15 23:01:32] Combining with fuzzy matches..."
#> [1] "[2025-01-15 23:01:32] Dropping duplicates..."
#> [1] "[2025-01-15 23:01:32] Checking for redundant name matches..."
#> [1] "[2025-01-15 23:01:32] Returning matched dataset!"

linkedOrgs_fuzzy
#>              orgnames_x   orgnames_y stringdist   minDist
#> 1                 apple   apple corp  0.5555556 0.5555556
#> 2                oracle   oracle inc  0.4444444 0.4444444
#> 3            enron inc.   oracle inc  0.7857143 0.7857143
#> 4            enron inc.        enron  0.5000000 0.5000000
#> 5 mcdonalds corporation   apple corp  0.8260870 0.8260870
#> 6 mcdonalds corporation   oracle inc  0.9200000 0.9200000
#> 7 mcdonalds corporation mcdonalds co  0.3888889 0.3888889

Here:

  • DistanceMeasure = "jaccard" controls the string distance measure used.
  • MaxDist = 0.5 sets the maximum “distance” for which a match is still acceptable. Adjust as needed.
  • algorithm = "fuzzy" ensures we use purely string-based matching.

ML Embedding Approach

The real power of LinkOrgs is the advanced ML-based approach that uses a learned embedding space. If you ran BuildBackend() and have a suitable conda environment, you can do:

linkedOrgs_ml <- LinkOrgs(x = x,
                          y = y,
                          by.x = "orgnames_x",
                          by.y = "orgnames_y",
                          MaxDist = 2.0,  # Euclidean distance in embedding space
                          algorithm = "ml",
                          conda_env = "LinkOrgsEnv", 
                          conda_env_required = T)
#> [1] "[2025-01-15 23:01:39] Re-building ML model..."
#> [1] "[2025-01-15 23:01:39] Loading LinkOrgs_Helpers.R..."
#> [1] "[2025-01-15 23:01:39] Loading JaxTransformer_Imports.R..."
#> [1] "[2025-01-15 23:01:42] Loading JaxTransformer_BuildML.R..."
#> [1] "[2025-01-15 23:01:47] Loading JaxTransformer_TrainDefine.R..."

#> [1] "[2025-01-15 23:02:47] Total parameter count: 29946062"
#> [1] "Default device backend: METAL"
#> [1] "[2025-01-15 23:02:47] Applying trained weights..."
#> [1] "[2025-01-15 23:02:47] Matching via name representations from ML models [v1]..."
#> [1] "[2025-01-15 23:02:47] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:02:47] 1"
#> [1] "[2025-01-15 23:02:47] Starting Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:02:48] Done with Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:02:48] Starting Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:02:51] Done with Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:03:08] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:03:08] 1"
#> [1] "[2025-01-15 23:03:09] Pre-processing strings..."
#> [1] "[2025-01-15 23:03:09] Searching for matches in the raw name space..."
#> [1] "[2025-01-15 23:03:09] Calibrating via AveMatchNumberPerAlias..."
#> [1] "[2025-01-15 23:03:09] Casting to matrix"
#> [1] "[2025-01-15 23:03:09] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [4, 1024]..."
#> Loading required package: Rcpp
#> Loading required package: RcppZiggurat
#> Loading required package: RcppParallel
#> 
#> Attaching package: 'RcppParallel'
#> The following object is masked from 'package:Rcpp':
#> 
#>     LdFlags
#> 
#> Rfast: 2.1.4
#>  ___ __ __ __ __    __ __ __ __ __ _             _               __ __ __ __ __     __ __ __ __ __ __   
#> |  __ __ __ __  |  |  __ __ __ __ _/            / \             |  __ __ __ __ /   /__ __ _   _ __ __\  
#> | |           | |  | |                         / _ \            | |                        / /          
#> | |           | |  | |                        / / \ \           | |                       / /          
#> | |           | |  | |                       / /   \ \          | |                      / /          
#> | |__ __ __ __| |  | |__ __ __ __           / /     \ \         | |__ __ __ __ _        / /__/\          
#> |    __ __ __ __|  |  __ __ __ __|         / /__ _ __\ \        |_ __ __ __ _   |      / ___  /           
#> |   \              | |                    / _ _ _ _ _ _ \                     | |      \/  / /       
#> | |\ \             | |                   / /           \ \                    | |         / /          
#> | | \ \            | |                  / /             \ \                   | |        / /          
#> | |  \ \           | |                 / /               \ \                  | |       / /          
#> | |   \ \__ __ _   | |                / /                 \ \     _ __ __ __ _| |      / /          
#> |_|    \__ __ __\  |_|               /_/                   \_\   /_ __ __ __ ___|      \/             team
#> 
#> Attaching package: 'Rfast'
#> The following object is masked from 'package:data.table':
#> 
#>     transpose
#> The following object is masked from 'package:dplyr':
#> 
#>     nth
#> [1] "[2025-01-15 23:03:09] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:03:09] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:03:09] Calibrated accept match dist threshold: 65.568377"
#> [1] "[2025-01-15 23:03:09] Casting to matrix"
#> [1] "[2025-01-15 23:03:09] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [4, 1024]..."
#> [1] "[2025-01-15 23:03:10] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:03:10] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:03:10] Combining with fuzzy matches..."
#> [1] "[2025-01-15 23:03:10] Dropping duplicates..."
#> [1] "[2025-01-15 23:03:10] Checking for redundant name matches..."
#> [1] "[2025-01-15 23:03:10] Returning matched dataset!"
                          
linkedOrgs_ml
#>               orgnames_x   orgnames_y stringdist   minDist
#> 1                  apple   apple corp  15.243541 15.243541
#> 2                  apple   oracle inc  45.353548 45.353548
#> 3                  apple        enron  53.024035 53.024035
#> 4                  apple mcdonalds co  60.608779 60.608779
#> 5                 oracle   apple corp  46.559494 46.559494
#> 6                 oracle   oracle inc   9.528285  9.528285
#> 7                 oracle        enron  36.213872 36.213872
#> 8                 oracle mcdonalds co  53.423096 53.423096
#> 9             enron inc.   apple corp  61.342775 61.342775
#> 10            enron inc.   oracle inc  41.032261 41.032261
#> 11            enron inc.        enron   8.355850  8.355850
#> 12            enron inc. mcdonalds co  56.752322 56.752322
#> 13 mcdonalds corporation   oracle inc  60.298200 60.298200
#> 14 mcdonalds corporation        enron  53.540465 53.540465
#> 15 mcdonalds corporation mcdonalds co  10.522919 10.522919
  • algorithm = "ml" instructs LinkOrgs() to use ML-based distance computations.
  • MaxDist = 2.0 is an approximate threshold in the learned embedding space. You can experiment with different values or rely on the built-in calibration AveMatchNumberPerAlias to set a more automatic threshold.

Network + ML (Ensemble)

The “ensemble” approach merges the strengths of ML embeddings and the LinkedIn network. You can access it by setting:

linkedOrgs_ensemble <- LinkOrgs(x = x,
                                y = y,
                                by.x = "orgnames_x",
                                by.y = "orgnames_y",
                                MaxDist_network = 2.0,
                                algorithm = "bipartite",  # or "markov"
                                DistanceMeasure = "ml",
                                conda_env = "LinkOrgsEnv", 
                                conda_env_required = T)
#> [1] "[2025-01-15 23:03:10] Re-building ML model..."
#> [1] "[2025-01-15 23:03:10] Loading LinkOrgs_Helpers.R..."
#> [1] "[2025-01-15 23:03:10] Loading JaxTransformer_Imports.R..."
#> [1] "[2025-01-15 23:03:11] Loading JaxTransformer_BuildML.R..."
#> [1] "[2025-01-15 23:03:11] Loading JaxTransformer_TrainDefine.R..."

#> [1] "[2025-01-15 23:04:10] Total parameter count: 29946062"
#> [1] "Default device backend: METAL"
#> [1] "[2025-01-15 23:04:10] Applying trained weights..."
#> [1] "[2025-01-15 23:04:10] Matching via name representations from ML models [v1]..."
#> [1] "[2025-01-15 23:04:10] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:04:10] 1"
#> [1] "[2025-01-15 23:04:10] Starting Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:04:11] Done with Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:04:11] Starting Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:04:14] Done with Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:04:28] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:04:28] 1"
#> [1] "[2025-01-15 23:04:35] Directory size: 583723 aliases"
#> [1] "[2025-01-15 23:05:18] Pre-processing strings..."
#> [1] "[2025-01-15 23:05:21] Searching for matches in the raw name space..."
#> [1] "[2025-01-15 23:05:21] Calibrating via AveMatchNumberPerAlias..."
#> [1] "[2025-01-15 23:05:21] Casting to matrix"
#> [1] "[2025-01-15 23:05:21] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [4, 1024]..."
#> [1] "[2025-01-15 23:05:22] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:22] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:22] Calibrated accept match dist threshold: 65.568377"
#> [1] "[2025-01-15 23:05:22] Casting to matrix"
#> [1] "[2025-01-15 23:05:22] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [4, 1024]..."
#> [1] "[2025-01-15 23:05:23] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:23] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:23] Searching for LinkedIn network matches"
#> [1] "[2025-01-15 23:05:23] Searching for network matches..."
#> [1] "[2025-01-15 23:05:23] Calibrating via AveMatchNumberPerAlias..."
#> [1] "[2025-01-15 23:05:23] Swapping for efficient use of vectorization..."
#> [1] "[2025-01-15 23:05:23] Casting to matrix"
#> [1] "[2025-01-15 23:05:23] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [1000, 1024]..."
#> [1] "[2025-01-15 23:05:24] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:24] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:24] Calibrated accept match dist threshold: 28.347942"
#> [1] "[2025-01-15 23:05:24] Casting to matrix"
#> [1] "[2025-01-15 23:05:25] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [562329, 1024]..."
#> [1] "[2025-01-15 23:05:32] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:32] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:33] De-duplicating merge among duplicates to canonical_id [keeping closest match]..."
#> [1] "[2025-01-15 23:05:33] Done deduplicating; final dimensions: 1846..."
#> [1] "[2025-01-15 23:05:33] Searching for network matches..."
#> [1] "[2025-01-15 23:05:33] Calibrating via AveMatchNumberPerAlias..."
#> [1] "[2025-01-15 23:05:33] Swapping for efficient use of vectorization..."
#> [1] "[2025-01-15 23:05:33] Casting to matrix"
#> [1] "[2025-01-15 23:05:33] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [1000, 1024]..."
#> [1] "[2025-01-15 23:05:34] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:34] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:34] Calibrated accept match dist threshold: 29.339919"
#> [1] "[2025-01-15 23:05:34] Casting to matrix"
#> [1] "[2025-01-15 23:05:35] Starting parallel Euclidean distance calculations with dim(x) = [4, 1024] and dim(y) = [562329, 1024]..."
#> [1] "[2025-01-15 23:05:41] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:05:41] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:05:41] De-duplicating merge among duplicates to canonical_id [keeping closest match]..."
#> [1] "[2025-01-15 23:05:41] Done deduplicating; final dimensions: 1765..."
#> [1] "[2025-01-15 23:05:42] Combining with fuzzy matches..."
#> [1] "[2025-01-15 23:05:42] Dropping duplicates..."
#> [1] "[2025-01-15 23:05:42] Checking for redundant name matches..."
#> [1] "[2025-01-15 23:05:42] Returning matched dataset!"

This can be more computationally heavy because:

  • Each observation in x and y is first matched to the LinkedIn directory.
  • We find the best canonical “community” or “alias ID.”
  • We also rely on the ML embeddings to refine link quality.

In small tasks, the difference might be minor, but in larger tasks with trickier alias issues, the ensemble often yields higher recall at the same precision.

Checking Match Performance via AssessMatchPerformance()

If you have a ground truth set of matches (often from manual or expert-coded data), you can easily assess how well your new algorithmic matches did with:

# Synthetic example: 
x_orgnames <- c("apple","oracle","enron inc.","mcdonalds corporation")
y_orgnames <- c("apple corp","oracle inc","enron","mcdonalds co")

x <- data.frame("orgnames_x"=x_orgnames)
y <- data.frame("orgnames_y"=y_orgnames)

# Suppose z is your matched dataset from LinkOrgs (or anything else)
z <- data.frame("orgnames_x"=x_orgnames[1:2], 
                "orgnames_y"=y_orgnames[1:2])

# Suppose z_true is your best ground truth:
z_true <- data.frame("orgnames_x"=x_orgnames, 
                     "orgnames_y"=y_orgnames)

# Evaluate:
PerformanceMatrix <- AssessMatchPerformance(x = x,
                                           y = y,
                                           z = z,
                                           z_true = z_true,
                                           by.x = "orgnames_x",
                                           by.y = "orgnames_y")

PerformanceMatrix
#>      TruePositives     FalsePositives     FalseNegatives      TrueNegatives 
#>                  2                  0                  2                 14 
#> MatchedDatasetSize 
#>                  2

The output is a named vector with:

These statistics are especially helpful when systematically adjusting match thresholds (e.g., MaxDist) to see how the false positive / false negative tradeoffs vary.

Downloading and Loading Remote Data: url2dt() and dropboxURL2downloadURL()

If you store large CSVs or ZIP files of data (possibly for replication) on Dropbox, the LinkOrgs package provides convenience functions for automatic downloading and importing into R:

# Download a zipped CSV from a Dropbox link and read it as a data.table
my_dt <- url2dt(url = "https://www.dropbox.com/s/iqf9ids77dckopf/Directory_LinkIt_bipartite_Embeddings.csv.zip?dl=0")

# Internally, url2dt uses dropboxURL2downloadURL() to fix the link so that
# it becomes a direct download (dl.dropboxusercontent.com).

# The function creates a temporary folder, downloads the file, unzips it, and returns a data.table.

Utility Functions

A few additional utility functions included in LinkOrgs:

Tips for Large-Scale Matching

Example: Bringing Everything Together

Below is a more fully featured example that shows a typical workflow using LinkOrgs. It uses fuzzy matching, then tries an ML-based approach, compares performances, and picks a threshold that balances false positives and false negatives:

library(LinkOrgs)

### 1. Suppose we have two large data frames:
###    lobbying_data (org name in col "lobby_org") 
###    finance_data  (org name in col "ticker_org")

# For illustration, use smaller synthetic ones
lobbying_data <- data.frame("lobby_org" = c("International Business Machs", 
                                            "microsft", 
                                            "Oracle inc."))
finance_data  <- data.frame("ticker_org"  = c("Microsoft Corporation", 
                                              "International Business Machines", 
                                              "Micron Tech", 
                                              "Oracle corp"))

### 2. Fuzzy matching approach:
fuzzy_matches <- LinkOrgs(
  x = lobbying_data,
  y = finance_data,
  by.x = "lobby_org",
  by.y = "ticker_org",
  algorithm = "fuzzy",
  DistanceMeasure = "jaccard",
  MaxDist = 0.7
)
#> [1] "[2025-01-15 23:05:49] Pre-processing strings..."
#> [1] "[2025-01-15 23:05:49] Searching for matches in the raw name space..."
#> [1] "[2025-01-15 23:05:49] Calibrating via AveMatchNumberPerAlias..."
#> Warning in e$fun(obj, substitute(ex), parent.frame(), e$data): already
#> exporting variable(s): split_list_x, DistanceMeasure, qgram, nCores,
#> ReturnProgress, x_tri_index, y_tri_index, MaxDist, x, by.x, y, by.y
#> [1] "[2025-01-15 23:05:50] Calibrated accept match dist threshold: 0.923077"
#> Warning in e$fun(obj, substitute(ex), parent.frame(), e$data): already
#> exporting variable(s): split_list_x, DistanceMeasure, qgram, nCores,
#> ReturnProgress, x_tri_index, y_tri_index, MaxDist, x, by.x, y, by.y
#> [1] "[2025-01-15 23:05:50] Combining with fuzzy matches..."
#> [1] "[2025-01-15 23:05:50] Dropping duplicates..."
#> [1] "[2025-01-15 23:05:50] Checking for redundant name matches..."
#> [1] "[2025-01-15 23:05:50] Returning matched dataset!"

### 3. ML approach (requires conda environment):
# BuildBackend(conda_env = "LinkOrgsEnv") # if not done yet
ml_matches <- LinkOrgs(
  x = lobbying_data,
  y = finance_data,
  by.x = "lobby_org",
  by.y = "ticker_org",
  algorithm = "ml",
  # We want an automatic threshold based on about 3 matches per name on average:
  AveMatchNumberPerAlias = 3,
  conda_env = "LinkOrgsEnv", 
  conda_env_required = T
)
#> [1] "[2025-01-15 23:05:50] Re-building ML model..."
#> [1] "[2025-01-15 23:05:50] Loading LinkOrgs_Helpers.R..."
#> [1] "[2025-01-15 23:05:50] Loading JaxTransformer_Imports.R..."
#> [1] "[2025-01-15 23:05:52] Loading JaxTransformer_BuildML.R..."
#> [1] "[2025-01-15 23:05:52] Loading JaxTransformer_TrainDefine.R..."

#> [1] "[2025-01-15 23:06:50] Total parameter count: 29946062"
#> [1] "Default device backend: METAL"
#> [1] "[2025-01-15 23:06:50] Applying trained weights..."
#> [1] "[2025-01-15 23:06:50] Matching via name representations from ML models [v1]..."
#> [1] "[2025-01-15 23:06:50] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:06:50] 1"
#> [1] "[2025-01-15 23:06:50] Starting Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:06:51] Done with Transformer block [depth 2; type word]..."
#> [1] "[2025-01-15 23:06:51] Starting Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:06:54] Done with Transformer block [depth 8; type alias]..."
#> [1] "[2025-01-15 23:07:09] Fraction complete, GetAliasRep_BigBatch() call:"
#> [1] "[2025-01-15 23:07:09] 1"
#> [1] "[2025-01-15 23:07:09] Pre-processing strings..."
#> [1] "[2025-01-15 23:07:09] Searching for matches in the raw name space..."
#> [1] "[2025-01-15 23:07:09] Calibrating via AveMatchNumberPerAlias..."
#> [1] "[2025-01-15 23:07:09] Swapping for efficient use of vectorization..."
#> [1] "[2025-01-15 23:07:10] Casting to matrix"
#> [1] "[2025-01-15 23:07:10] Starting parallel Euclidean distance calculations with dim(x) = [3, 1024] and dim(y) = [4, 1024]..."
#> [1] "[2025-01-15 23:07:10] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:07:10] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:07:10] Calibrated accept match dist threshold: 59.933774"
#> [1] "[2025-01-15 23:07:10] Casting to matrix"
#> [1] "[2025-01-15 23:07:10] Starting parallel Euclidean distance calculations with dim(x) = [3, 1024] and dim(y) = [4, 1024]..."
#> [1] "[2025-01-15 23:07:11] Done with Euclidean distance calculations..."
#> [1] "[2025-01-15 23:07:11] Wrapping up call to pDistMatch_euclidean()..."
#> [1] "[2025-01-15 23:07:11] Combining with fuzzy matches..."
#> [1] "[2025-01-15 23:07:11] Dropping duplicates..."
#> [1] "[2025-01-15 23:07:11] Checking for redundant name matches..."
#> [1] "[2025-01-15 23:07:11] Returning matched dataset!"

### 4. Evaluate with a small ground truth set (hypothetical):
z_true <- data.frame(
  "lobby_org" = c("International Business Machs", "microsft", "Oracle inc."),
  "ticker_org" = c("International Business Machines", "Microsoft Corporation", 
                   "Oracle corp")
)

Perf_fuzzy <- AssessMatchPerformance(
  x = lobbying_data,
  y = finance_data,
  z = fuzzy_matches,
  z_true = z_true,
  by.x = "lobby_org",
  by.y = "ticker_org"
)

Perf_ml <- AssessMatchPerformance(
  x = lobbying_data,
  y = finance_data,
  z = ml_matches,
  z_true = z_true,
  by.x = "lobby_org",
  by.y = "ticker_org"
)

Perf_fuzzy
#>      TruePositives     FalsePositives     FalseNegatives      TrueNegatives 
#>                  3                  3                  0                  6 
#> MatchedDatasetSize 
#>                  6
Perf_ml
#>      TruePositives     FalsePositives     FalseNegatives      TrueNegatives 
#>                  3                  7                  0                  2 
#> MatchedDatasetSize 
#>                 10

# Typically, the ML approach will better match "microsft" to "Microsoft Corporation"
# and "International Business Machs" to "International Business Machines".

Conclusion

You’ve seen how to install the LinkOrgs package, build a conda environment for advanced ML matching, run the main function with various algorithms, and evaluate your matches.

Key points to remember:

For more information, check out the package documentation