file_db Directory Reference

Description

C++ API for the Data Management System (DMS)

It is implemented as the FileCatalogue class (on C++) using CURL for calling the functionality of the File Catalogue REST API. It provides functions to work with metadata of BM@N data files, to retrieve lists of files that meet specific criteria, and upload/download files to/from the FileCatalogue of the experiment. User authorization is performed through BM@N Keycloak system (OpenID protocol). The user must be in one of the access groups: bmnfilereader/bmnfilewriter/bmnfileadmin.

Contents

File / directory	Description
FileCatalogue	Interface to the File Catalogue: search, get, add, update, delete and check the file and directory metadata, upload and download the files, get the physical file names
FileInfo	Metadata of one file
FileCondition	Condition (attribute, comparison, value) to search for the files
file_db_settings.h	Settings of the REST API service and the user account
test_FC.cpp	test-FC program testing the functions of the interface
examples	Examples of getting the file lists, file metadata and files

The interface is built into the FileDb library; it is a git submodule developed in the nica_db/dms_api repository at git.jinr.ru.

Prerequisites

1. CURL library is required to work with the interface. To install the library for RedHat-based OS (eg, CentOS, Scientific Linux, AlmaLinux), run the following command:

   sudo yum install libcurl-devel

   for Debian-based OS (eg, Ubuntu, Debian):

   sudo apt-get install libcurl4-openssl-dev

2. Set your login|password of the JINR account in the corresponding variables of the 'file_db_settings.h' file.

Usage

Get a list of files that meet certain criteria (metadata)

1. Create one or more FileCondition objects with necessary criteria (energy >= 2 in the example below).

   enum FileCondition::searchAttributes attr = FileCondition::kEnergy;
   enum FileCondition::matchCriteria criteria = FileCondition::kGreaterOrEqual;
   int condValue = 2;
   std::unique_ptr<FileCondition> fcond = std::make_unique<FileCondition>(attr, criteria, condValue);

2. Add all the FileCondition objects to a vector.

   vector<unique_ptr<FileCondition>> fconds;
   fconds.push_back(std::move(fcond));

3. Call GetFileList() function and pass the vector to the function, include another vector as return parameter.

   FileCatalogue* FC = new FileCatalogue();
   vector<string> filepaths;
   int res = FC->GetFileList(std::move(fconds), filepaths);
   if (res)
       cout << "Error: " << GetErrorString(-res) << endl;
   else {
       cout << " Here's your filepaths: " << endl;
       for (auto i : filepaths)
           cout << i << endl;
   }

Get metadata for a given file

FileCatalogue *FC = new FileCatalogue();
string LFN = "/bmn.nica.jinr/exp/raw/run9/run_Top_9639_ev0_p24.data";
FileInfo fi;
int res = FC->GetFileInfo(LFN, fi);
if (res)
    cout << "Error: " << GetErrorString(-res) << endl;
else
    fi.Print();

Download necessary files from the File Catalogue by its LFN to local path or to xrootd URL.

FileCatalogue *FC = new FileCatalogue();
string LFN = "/bmn.nica.jinr/user/i/isjiron/newone3.data";
string save_path = "/Users/beeleph/tmp/plusdownload.data";
int res = FC->GetFile(LFN, save_path);

OR

FileCatalogue *FC = new FileCatalogue();
string LFN = "/bmn.nica.jinr/user/i/isjiron/newone3.data";
string save_path = "root://isjiron@eos.jinr.ru:1094//eos/nica/bmn/dirac/bmn.nica.jinr/user/i/isjiron/data_via_xrd_via_cpp.data";
int res = FC->GetFile(LFN, save_path);

Upload file and register at FileCatalogue

FileCatalogue *FC = new FileCatalogue();
string LFN = "/bmn.nica.jinr/user/i/isjiron/big2.data";
string filepath = "/Users/beeleph/tmp/bigfile.data";
int res = FC->AddFile(filepath, LFN);

Get PFNs (Physical File Name) of file by it's LFN (Logical File Name)

FileCatalogue *FC = new FileCatalogue();
string LFN = "/bmn.nica.jinr/user/i/isjiron/newone3.data";
vector<string> *pfns = new vector<string>;
int res = FC->GetPFNs(LFN, *pfns);
if (res)
    cout << "Error: " << GetErrorString(-res) << endl;
else {
    cout << " Here's your PFNs: " << endl;
    for (auto i : *pfns)
        cout << i << endl;
}

Roadmap

Support

Zhironkin Igor Sergeevich jiron.nosp@m.kin@.nosp@m.jinr..nosp@m.ru isjir.nosp@m.onn@.nosp@m.gmail.nosp@m..com

Directory dependency graph for file_db:

Directories
	examples
	File Catalogue examples

Files
	file_db_settings.h
	FileCatalogue.cxx
	FileCatalogue.h
	test_FC.cpp