Magic ETL v2 Scripting Tile Environments

​

Intro

​

Default Supported Libraries in Magic ETL v2

​

Python

• absl-py 1.0.0 - Abseil Python Common Libraries
• asn1crypto 1.5.1 - Fast ASN.1 parser and serializer with definitions for private keys, public keys, certificates, CRL, OCSP, CMS, PKCS#3, PKCS#7, PKCS#8, PKCS#12, PKCS#5, X.509 and TSP
• astor 0.8.1 - Read/rewrite/write Python ASTs
• blinker 1.4 - Fast, simple object-to-object and broadcast signaling
• certifi 2021.10.8 - Python package for providing Mozilla’s CA Bundle.
• cookies 2.2.1 - Friendlier RFC 6265-compliant cookie parser/renderer
• cryptography 36.0.2 - cryptography is a package which provides cryptographic recipes and primitives to Python developers.
• decorator 5.1.1 - Better living through Python with decorators
• domomagic 0.3 - Python integration with Domo Magic ETL
• fuzzywuzzy 0.18.0 - Fuzzy string matching in python
• gast 0.4.0 - Python AST that abstracts the underlying Python version
• grpcio 1.42.0 - HTTP/2-based RPC framework
• h5py 3.6.0 - Read and write HDF5 files from Python
• idna 3.3 - Internationalized Domain Names in Applications (IDNA)
• joblib 1.1.0 - Lightweight pipelining: using Python functions as pipeline jobs.
• keras 2.7.0 - Artificial neural networks. Acts as an interface for the TensorFlow library.
• Markdown 3.3.6 - Python implementation of Markdown.
• nltk 3.6.7 - Natural Language Toolkit
• numpy 1.22.3 - NumPy: array processing for numbers, strings, records, and objects.
• pandas 1.4.1 - Powerful data structures for data analysis, time series,and statistics
• patsy 0.5.2 - A Python package for describing statistical models and for building design matrices.
• Pillow 9.0.1 - Python Imaging Library (Fork)
• protobuf 3.19.4 - Protocol Buffers
• python-dateutil 2.8.2 - Extensions to the standard Python datetime module
• python-Levenshtein 0.12.2 - Python extension for computing string edit distances and similarities.
• pytz 2022.1 - World timezone definitions, modern and historical
• requests 2.27.1 - HTTP for Humans
• scikit-learn 1.0.2 - A set of python modules for machine learning and data mining
• scipy 1.8.0 - SciPy: Scientific Library for Python
• scour 0.38.2 - Scour SVG Optimizer
• six 1.16.0 - Python 2 and 3 compatibility utilities
• sklearn-pandas 2.2.0 - Pandas integration with sklearn
• tensorboard 2.6.0 - TensorBoard lets you watch Tensors Flow
• tensorflow 2.7.0 - TensorFlow is an open source machine learning framework for everyone.
• termcolor 1.1.0 - ANSII Color formatting for output in terminal.
• torch 1.11.0 - Tensors and Dynamic neural networks in Python with strong GPU acceleration
• torchvision 0.12.0 - image and video datasets and models for torch deep learning
• twython 3.9.1 - Actively maintained, pure Python wrapper for the Twitter API. Supports both normal and streaming Twitter APIs
• urllib3 1.26.9 - HTTP library with thread-safe connection pooling, file post, and more.
• Werkzeug 2.0.3 - The comprehensive WSGI web application library.
• xgboost 1.5.1 - Scalable, distributed gradient-boosted decision tree (GBDT) machine learning library

​

R

• abind 1.4-5 - Combine Multidimensional Arrays
• acepack 1.4.1 - ace() and avas() for selecting regression transformations
• AER 1.2-9 - Applied Econometrics with R
• arm 1.12_2 - Data Analysis Using Regression and Multilevel/Hierarchical
• arules 1.7_3 - Mining Association Rules and Frequent Itemsets
• assertthat 0.2.1 - Easy Pre and Post Assertions
• backports 1.4.1 - Reimplementations of Functions Introduced Since R-3.0.0
• base64enc 0.1-3 - Tools for base64 encoding
• base 4.1.2 - The R Base Package (Pre-loaded and attached package)
• Boruta 7.0.0 - An all relevant feature selection wrapper algorithm.
• BH 1.78.0-0 - Boost C++ Header Files
• bindrcpp 0.2.2 - An ‘Rcpp’ Interface to Active Bindings
• bindr 0.1.1 - Parametrized Active Bindings
• bit64 4.0.5 - A S3 Class for Vectors of 64bit Integers
• bit 4.0.4 - A Class for Vectors of 1-Bit Booleans
• bitops 1.0-7 - Bitwise Operations
• blob 1.2.2 - A Simple S3 Class for Representing Vectors of Binary Data
• boot 1.3-28 - Bootstrap Functions (Originally by Angelo Canty for S)
• brew 1.0-7 - Templating Framework for Report Generation
• broom 0.7.12 - Convert Statistical Analysis Objects into Tidy Tibbles
• car 3.0_12 - Companion to Applied Regression
• caret 6.0-91 - Classification and Regression Training
• caTools 1.18.2 - Tools: moving window statistics, GIF, Base64, ROC AUC, etc.
• CerioliOutlierDetection 1.1.9 - Outlier Detection Using the Iterated RMCD Method of Cerioli
• checkmate 2.0.0 - Fast and Versatile Argument Checks
• chron 2.3-52 - Chronological Objects which can Handle Dates and Times
• class 7.3-20 - Functions for Classification
• cli 3.2.0 - Helpers for Developing Command Line Interfaces
• cluster 2.1.2 - Finding Groups in Data: Cluster Analysis Extended Rousseeuw et
• coda 0.19-4 - Output Analysis and Diagnostics for MCMC
• codetools 0.2-18 - Code Analysis Tools for R
• coin 1.4_2 - Conditional Inference Procedures in a Permutation Test Framework
• colorspace 2.0_3 - Color Space Manipulation
• commonmark 1.8.0 - High Performance CommonMark and Github Markdown Rendering in R
• corpcor 1.6.10 - Efficient Estimation of Covariance and (Partial) Correlation
• crayon 1.5.0 - Colored Terminal Output
• Cubist 0.4.0 - Rule- And Instance-Based Regression Modeling
• curl 4.3.2 - A Modern and Flexible Web Client for R
• CVST 0.2-3 - Fast Cross-Validation via Sequential Testing
• cvTools 0.3.2 - Cross-validation tools for regression models
• data.table 1.14.2 - Extension of data.frame
• DBI 1.1.2 - R Database Interface
• ddalpha 1.3.12 - Depth-Based Classification and Calculation of Data Depth
• DEoptimR 1.0-10 - Differential Evolution Optimization in Pure R
• DMwR 0.4.1 - Functions and data accompanying the book ‘Data Mining with R, learning with case studies’ by Luis Torgo, CRC Press 2010.
• desc 1.4.1 - Manipulate DESCRIPTION Files
• dichromat 2.0-0 - Color Schemes for Dichromats
• digest 0.6.29 - Create Compact Hash Digests of R Objects
• dimRed 0.2.5 - A Framework for Dimensionality Reduction
• diptest 0.76_0 - Hartigan’s Dip Test Statistic for Unimodality - Corrected
• domomagic 0.4 - R integration with Domo Magic ETL
• dplyr 1.0.8 - A Grammar of Data Manipulation
• DRR 0.0.4 - Dimensionality Reduction via Regression
• dummies 1.5.6 - Create dummy/indicator variables flexibly and efficiently
• e1071 1.7_9 - Misc Functions of the Department of Statistics, Probability
• evaluate 0.15 - Parsing and Evaluation Tools that Provide More Details than the
• fansi 1.0.2 - ANSI Control Sequence Aware String Functions
• flexclust 1.4_0 - Flexible Cluster Algorithms
• flexmix 2.3-17 - Flexible Mixture Modeling
• foreach 1.5.2 - Provides Foreach Looping Construct for R
• forecast 8.16 - Forecasting Functions for Time Series and Linear Models
• forcats 0.5.1 - Tools for working with categorical variables
• foreign 0.8-82 - Read Data Stored by ‘Minitab’, ‘S’, ‘SAS’, ‘SPSS’, ‘Stata’,
• Formula 1.2-4 - Extended Model Formulas
• fpc 2.2_9 - Flexible Procedures for Clustering
• fracdiff 1.5_1 - Fractionally differenced ARIMA aka ARFIMA(p,d,q) models
• gains 1.2 - Constructs gains tables and lift charts for prediction algorithms.
• gbm 2.1.8 - Generalized Boosted Regression Models
• gdata 2.18.0 - Various R Programming Tools for Data Manipulation
• GGally 2.1.2 - Extension to ‘ggplot2’
• ggplot2 3.3.5 - Create Elegant Data Visualisations Using the Grammar of Graphics
• git2r 0.30.1 - Provides Access to Git Repositories
• glmnet 4.1_2 - Lasso and Elastic-Net Regularized Generalized Linear Models
• glue 1.6.2 - Interpreted String Literals
• Gmedian 1.2.6 - Geometric Median, k-Median Clustering and Robust Median PCA
• gower 1.0.0 - Gower’s Distance
• gridExtra 2.3 - Miscellaneous Functions for Grid Graphics
• gtable 0.3.0 - Arrange ‘Grobs’ in Tables
• gtools 3.9.2 - Various R Programming Tools
• highr 0.9 - Syntax Highlighting for R Source Code
• Hmisc 4.6_0 - Harrell Miscellaneous
• hms 1.1.1 - Pretty Time of Day
• htmlTable 2.4.0 - Advanced Tables for Markdown/HTML
• htmltools 0.5.2 - Tools for HTML
• htmlwidgets 1.5.4 - HTML Widgets for R
• httr 1.4.2 - Tools for Working with URLs and HTTP
• hunspell 3.0.1 - High-Performance Stemmer, Tokenizer, and Spell Checker
• ipred 0.9-12 - Improved Predictors
• ISOcodes 2022.01.10 - Selected ISO Codes
• iterators 1.0.14 - Provides Iterator Construct for R
• janeaustenr 0.1.5 - Jane Austen’s Complete Novels
• janitor 2.1.0 - Simple functions for examining and cleaning dirty data.
• jsonlite 1.8.0 - A Robust, High Performance JSON Parser and Generator for R
• kernlab 0.9-29 - Kernel-Based Machine Learning Lab
• KernSmooth 2.23-20 - Functions for Kernel Smoothing Supporting Wand & Jones (1995)
• KMsurv 0.1-5 - Data sets from Klein and Moeschberger (1997), Survival Analysis
• knitr 1.37 - A General-Purpose Package for Dynamic Report Generation in R
• labeling 0.4.2 - Axis Labeling
• laeken 0.5.2 - Estimation of indicators on social exclusion and poverty
• lattice 0.20-42 - Trellis Graphics for R
• latticeExtra 0.6-29 - Extra Graphical Utilities Based on Lattice
• lava 1.6.10 - Latent Variable Models
• lazyeval 0.2.2 - Lazy (Non-Standard) Evaluation
• leaps 3.1 - regression subset selection
• littler 0.3.15 - R at the Command-Line via ‘r’
• lme4 1.1-28 - Linear Mixed-Effects Models using ‘Eigen’ and S4
• lmtest 0.9-40 - Testing Linear Regression Models
• lubridate 1.8.0 - Make Dealing with Dates a Little Easier
• magrittr 2.0.2 - A Forward-Pipe Operator for R
• maptools 1.1_3 - Tools for Handling Spatial Objects
• markdown 1.1 - ‘Markdown’ Rendering for R
• MASS 7.3-55 - Support Functions and Datasets for Venables and Ripley’s MASS
• matrixcalc 1.0-5 - Collection of functions for matrix calculations
• Matrix 1.4_0 - Sparse and Dense Matrix Classes and Methods
• MatrixModels 0.5_0 - Modelling with Sparse And Dense Matrices
• maxLik 1.5_2 - Maximum Likelihood Estimation and Related Tools
• mclust 5.4.9 - Gaussian Mixture Modelling for Model-Based Clustering,
• mcmc 0.9-7 - Markov Chain Monte Carlo
• MCMCpack 1.6_1 - Markov Chain Monte Carlo (MCMC) Package
• memoise 2.0.1 - Memoisation of Functions
• mice 3.14.0 - Multivariate Imputation by Chained Equations
• mi 1.0 - Missing Data Imputation and Model Checking
• mime 0.12 - Map Filenames to MIME Types
• minqa 1.2.4 - Derivative-free optimization algorithms by quadratic
• miscTools 0.6-26 - Miscellaneous Tools and Utilities
• mitools 2.4 - Tools for multiple imputation of missing data
• mlbench 2.1-3 - A collection of artificial and real-world machine learning benchmark problems.
• mlogit 1.1_1 - Multinomial Logit Models
• mnormt 2.0.2 - The Multivariate Normal and t Distributions
• ModelMetrics 1.2.2.2 - Rapid Calculation of Model Metrics
• modeltools 0.2-23 - Tools and Classes for Statistical Models
• moments 0.14 - Moments, cumulants, skewness, kurtosis and related tests
• multcomp 1.4-18 - Simultaneous Inference in General Parametric Models
• munsell 0.5.0 - Utilities for Using Munsell Colours
• mvoutlier 2.1.1 - Multivariate Outlier Detection Based on Robust Methods
• mvtnorm 1.1_3 - Multivariate Normal and t Distributions
• NADA 1.6-1.1 - Nondetects and Data Analysis for Environmental Data
• naivebayes 0.9.7 - High Performance Implementation of the Naive Bayes Algorithm
• neuralnet 1.44.2 - Training of Neural Networks
• nlme 3.1-155 - Linear and Nonlinear Mixed Effects Models
• nloptr 1.2.2.3 - R Interface to NLopt
• NLP 0.2_1 - Natural Language Processing Infrastructure
• nnet 7.3-17 - Feed-Forward Neural Networks and Multinomial Log-Linear Models
• numDeriv 2016.8-1.1 - Accurate Numerical Derivatives
• openssl 2.0.0 - Toolkit for Encryption, Signatures and Certificates Based on
• outliers 0.14 - Tests for outliers
• party 1.3-9 - A Laboratory for Recursive Partytioning
• pbkrtest 0.5.1 - Parametric Bootstrap and Kenward Roger Based Methods for Mixed
• pcaPP 1.9-74 - Robust PCA by Projection Pursuit
• penalized 0.9-51 - L1 (Lasso and Fused Lasso) and L2 (Ridge) Penalized Estimation
• pillar 1.7.0 - Coloured Formatting for Columns
• pkgconfig 2.0.3 - Private Configuration for ‘R’ Packages
• pkgKitten 0.2.2 - Create Simple Packages Which Do not Upset R Package Checks
• plogr 0.2.0 - The ‘plog’ C++ Logging Library
• pls 2.8-0 - Partial Least Squares and Principal Component Regression
• plyr 1.8.6 - Tools for Splitting, Applying and Combining Data
• pmml 2.5.2 - Predictive Model Markup Language support.
• polspline 1.1.19 - Polynomial Spline Routines
• prabclus 2.3_2 - Functions for Clustering of Presence-Absence, Abundance and
• praise 1.0.0 - Praise Users
• prettyunits 1.1.1 - Pretty, Human Readable Formatting of Quantities
• prodlim 2019.11.13 - Product-Limit Estimation for Censored Event History Analysis
• progress 1.2.2 - Terminal Progress Bars
• pryr 0.1.5 - Tools for Computing on the Language
• psych 2.2.3 - Procedures for Psychological, Psychometric, and Personality
• purrr 0.3.4 - Functional Programming Tools
• quadprog 1.5-8 - Functions to solve Quadratic Programming Problems.
• quantmod 0.4.18 - Quantitative Financial Modelling Framework
• quantreg 5.88 - Quantile Regression
• R6 2.5.1 - Classes with Reference Semantics
• randomForest 4.6-14 - Breiman and Cutler’s Random Forests for Classification and
• randomForestSRC 3.0.2 - Random Forests for Survival, Regression, and Classification
• RColorBrewer 1.1-2 - ColorBrewer Palettes
• RcppArmadillo 0.10.8.1.0 - ‘Rcpp’ Integration for the ‘Armadillo’ Templated Linear Algebra
• Rcpp 1.0.8.3 - Seamless R and C++ Integration
• RcppEigen 0.3.3.9.1 - ‘Rcpp’ Integration for the ‘Eigen’ Templated Linear Algebra
• RcppRoll 0.3.0 - Efficient Rolling / Windowed Operations
• rcurl 1.98_1.6 - RCurl
• recipes 0.2.0 - Preprocessing Tools to Create Design Matrices
• rebus 0.1_3 - Regular Expression Builder
• REEMtree 0.90.4 - Regression Trees with Random Effects for Longitudinal (Panel)
• ROCR 1.0-11 - Tool for creating cutoff-parameterized 2D performance curves.
• ROSE 0.0-4 - Functions to deal with binary classification problems in the presence of imbalanced classes.
• relaimpo 2.2-6 - Relative Importance of Regressors in Linear Models
• reshape2 1.4.4 - Flexibly Reshape Data: A Reboot of the Reshape Package
• reshape 0.8.8 - Flexibly Reshape Data
• rgeos 0.5_9 - Interface to Geometry Engine - Open Source (‘GEOS’)
• rjson 0.2.21 - JSON for R
• rlang 1.0.2 - Functions for Base Types and Core R and ‘Tidyverse’ Features
• rms 6.2-0 - Regression Modeling Strategies
• robCompositions 2.3.1 - Robust Estimation for Compositional Data
• robustbase 0.93-9 - Basic Robust Statistics
• roxygen2 7.1.2 - In-Line Documentation for R
• rpart 4.1.16 - Recursive Partitioning and Regression Trees
• RPMM 1.25 - Recursively Partitioned Mixture Model
• rprojroot 2.0.2 - Finding Files in Project Subdirectories
• rrcov 1.6_2 - Scalable Robust Estimators with High Breakdown Point
• RSNNS 0.4-14 - Neural Networks using the Stuttgart Neural Network Simulator
• RSpectra 0.16_0 - Solvers for Large-Scale Eigenvalue and SVD Problems
• RSQLite 2.2.8 - ‘SQLite’ Interface for R
• rstudioapi 0.13 - Safely Access the RStudio API
• sandwich 3.0_1 - Robust Covariance Matrix Estimators
• scales 1.1.1 - Scale Functions for Visualization
• sem 3.1-14 - Structural Equation Models
• sfsmisc 1.1-12 - Utilities from ‘Seminar fuer Statistik’ ETH Zurich
• sgeostat 1.0-27 - An Object-Oriented Framework for Geostatistical Modeling in S+
• slam 0.1-50 - Sparse Lightweight Arrays and Matrices
• SnowballC 0.7.0 - Snowball stemmers based on the C libstemmer UTF-8 library
• SparseM 1.81 - Sparse Linear Algebra
• spatial 7.3-15 - Functions for Kriging and Point Pattern Analysis
• sp 1.4_6 - Classes and Methods for Spatial Data
• splines2 0.4.5 - Regression Spline Functions and Classes
• sROC 0.1-2 - Nonparametric Smooth ROC Curves for Continuous Data
• statmod 1.4.36 - Statistical Modeling
• sticky 0.5.6.1 - Persist Attributes Across Data Operations
• stopwords 2.3 - Multilingual Stopword Lists
• stringdist 0.9.8 - Approximate String Matching and String Distance Functions
• stringi 1.7.6 - Character String Processing Facilities
• stringr 1.4.0 - Simple, Consistent Wrappers for Common String Operations
• strucchange 1.5-2 - Testing, Monitoring, and Dating Structural Changes
• survey 4.1_1 - Analysis of Complex Survey Samples
• survival 3.3_1 - Survival Analysis
• tcltk2 1.2_11 - Tcl/Tk Interface
• TH.data 1.1_0 - TH’s Data Archive
• tibble 3.1.6 - Simple Data Frames
• tidymodels 0.2.0 - A collection of packages for modeling and machine learning using tidyverse principles.
• tidyr 1.2.0 - Easily Tidy Data with ‘spread()’ and ‘gather()’ Functions
• tidyselect 1.1.2 - Select from a Set of Strings
• tidytext 0.3.2 - Text Mining using ‘dplyr’, ‘ggplot2’, and Other Tidy Tools
• tidyverse 1.3.1 - An opinionated collection of R packages designed for data science
• timeDate 3043.102 - Rmetrics - Chronological and Calendar Objects
• timeSeries 3062.100 - Rmetrics - Financial Time Series Objects
• tm 0.7-8 - Text Mining Package
• tokenizers 0.2.1 - Fast, Consistent Tokenization of Natural Language Text
• tree 1.0-41 - Classification and Regression Trees
• trimcluster 0.1-5 - Cluster analysis with trimming
• truncnorm 1.0-8 - Truncated Normal Distribution
• truncreg 0.2-5 - Truncated Gaussian Regression Models
• tseries 0.10-49 - Time Series Analysis and Computational Finance
• tsoutliers 0.6-8 - Detection of Outliers in Time Series
• TTR 0.24_3 - Technical Trading Rules
• utf8 1.2.2 - Unicode Text Processing
• vcd 1.4-9 - Visualizing Categorical Data
• VIM 6.1.1 - Visualization and Imputation of Missing Values
• VSURF 1.1.0 - Three steps variable selection procedure based on random forests.
• viridis 0.6.2 - Default Color Maps from ‘matplotlib’
• viridisLite 0.4.0 - Default Color Maps from ‘matplotlib’ (Lite Version)
• whisker 0.4 - {} for R, logicless templating
• withr 2.5.0 - Run Code ‘With’ Temporarily Modified Global State
• xgboost 1.5.1 - Extreme Gradient Boosting
• xml2 1.3.3 - Parse XML
• xts 0.12.1 - eXtensible Time Series
• yaml 2.3.5 - Methods to Convert R Data to YAML and Back
• zCompositions 1.4.0 - Treatment of Zeros and Nondetects in Compositional Data Sets
• zoo 1.8-9 - S3 Infrastructure for Regular and Irregular Time Series (Z’s