# Model¶

This module is designed to include machine-learning models for interpolating energies and forces from either an atom-centered or image-centered fingerprint description.

## Model¶

class amp.model.LossFunction(energy_coefficient=1.0, force_coefficient=0.04, convergence=None, parallel=None, overfit=0.0, raise_ConvergenceOccurred=True, log_losses=True, d=None)[source]

Basic loss function, which can be used by the model.get_loss method which is required in standard model classes.

This version is pure python and thus will be slow compared to a fortran/parallel implementation.

If parallel is None, it will pull it from the model itself. Only use this keyword to override the model’s specification.

Also has parallelization methods built in.

See self.default_parameters for the default values of parameters specified as None.

Parameters: energy_coefficient (float) – Coefficient of the energy contribution in the loss function. force_coefficient (float) – Coefficient of the force contribution in the loss function. Can set to None as shortcut to turn off force training. convergence (dict) – Dictionary of keys and values defining convergence. Keys are ‘energy_rmse’, ‘energy_maxresid’, ‘force_rmse’, and ‘force_maxresid’. If ‘force_rmse’ and ‘force_maxresid’ are both set to None, force training is turned off and force_coefficient is set to None. parallel (dict) – Parallel configuration dictionary. Will pull from model itself if not specified. overfit (float) – Multiplier of the weights norm penalty term in the loss function. raise_ConvergenceOccurred (bool) – If True will raise convergence notice. log_losses (bool) – If True will log the loss function value in the log file else will not. d (None or float) – If d is None, both loss function and its gradient are calculated analytically. If d is a float, then gradient of the loss function is calculated by perturbing each parameter plus/minus d.
attach_model(model, images=None, fingerprints=None, fingerprintprimes=None)[source]

Attach the model to be used to the loss function. hashed images, fingerprints and fingerprintprimes can optionally be specified; this is typically for use in parallelization.

Parameters: model (object) – Class representing the regression model. images (dict) – Dictionary of hashed images to train on. fingerprints (dict) – Fingerprints of images to train on. fingerprintprimes (dict) – Fingerprint derivatives of images to train on.
calculate_loss(parametervector, lossprime)[source]

Method that calculates the loss, derivative of the loss with respect to parameters (if requested), and max_residual.

Parameters: parametervector (list) – Parameters of the regression model in the form of a list. lossprime (bool) – If True, will calculate and return dloss_dparameters, else will only return zero for dloss_dparameters.
check_convergence(loss, energy_loss, force_loss, energy_maxresid, force_maxresid)[source]

Check convergence

Checks to see whether convergence is met; if it is, raises ConvergenceException to stop the optimizer.

Parameters: loss (float) – Value of the loss function. energy_loss (float) – Value of the energy contribution of the loss function. force_loss (float) – Value of the force contribution of the loss function. energy_maxresid (float) – Maximum energy residual. force_maxresid (float) – Maximum force residual.
default_parameters = {'convergence': {'energy_rmse': 0.001, 'force_rmse': None, 'energy_maxresid': None, 'force_maxresid': None}}
get_loss(parametervector, lossprime)[source]

Returns the current value of the loss function for a given set of parameters, or, if the energy is less than the energy_tol raises a ConvergenceException.

Parameters: parametervector (list) – Parameters of the regression model in the form of a list. lossprime (bool) – If True, will calculate and return dloss_dparameters, else will only return zero for dloss_dparameters.
process_parallels(vector, server, n_pids)[source]
Parameters: vector (list) – Parameters of the regression model in the form of a list. server (object) – Master session of parallel processing. processes (list of objects) – Worker sessions for parallel processing.
class amp.model.Model[source]

Bases: object

Class that includes common methods between different models.

calculate_dEnergy_dParameters(fingerprints)[source]

Calculates a list of floats corresponding to the derivative of model-predicted energy of an image with respect to model parameters.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values.
calculate_dForces_dParameters(fingerprints, fingerprintprimes)[source]

Calculates an array of floats corresponding to the derivative of model-predicted atomic forces of an image with respect to model parameters.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values. fingerprintprimes (dict) – Dictionary with images hashs as keys and the corresponding fingerprint derivatives as values.
calculate_energy(fingerprints)[source]

Calculates the model-predicted energy for an image, based on its fingerprint.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values.
calculate_forces(fingerprints, fingerprintprimes)[source]

Calculates the model-predicted forces for an image, based on derivatives of fingerprints.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values. fingerprintprimes (dict) – Dictionary with images hashs as keys and the corresponding fingerprint derivatives as values.
calculate_numerical_dEnergy_dParameters(fingerprints, d=1e-05)[source]

Evaluates dEnergy_dParameters using finite difference.

This will trigger two calls to calculate_energy(), with each parameter perturbed plus/minus d.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values. d (float) – The amount of perturbation in each parameter.
calculate_numerical_dForces_dParameters(fingerprints, fingerprintprimes, d=1e-05)[source]

Evaluates dForces_dParameters using finite difference. This will trigger two calls to calculate_forces(), with each parameter perturbed plus/minus d.

Parameters: fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values. fingerprintprimes (dict) – Dictionary with images hashs as keys and the corresponding fingerprint derivatives as values. d (float) – The amount of perturbation in each parameter.
log

Method to set or get a logger. Should be an instance of amp.utilities.Logger.

Parameters: log (Logger object) – Write function at which to log data. Note this must be a callable function.
tostring()[source]

Returns an evaluatable representation of the calculator that can be used to re-establish the calculator.

amp.model.calculate_fingerprints_range(fp, images)[source]

Calculates the range for the fingerprints corresponding to images, stored in fp. fp is a fingerprints object with the fingerprints data stored in a dictionary-like object at fp.fingerprints. (Typically this is a .utilties.Data structure.) images is a hashed dictionary of atoms for which to consider the range.

In image-centered mode, returns an array of (min, max) values for each fingerprint. In atom-centered mode, returns a dictionary of such arrays, one per element.

amp.model.ravel_data(train_forces, mode, images, fingerprints, fingerprintprimes)[source]

Reshapes data of images into lists.

Parameters: train_forces (bool) – Determining whether forces are also trained or not. mode (str) – Can be either ‘atom-centered’ or ‘image-centered’. images (list or str) – List of ASE atoms objects with positions, symbols, energies, and forces in ASE format. This is the training set of data. This can also be the path to an ASE trajectory (.traj) or database (.db) file. Energies can be obtained from any reference, e.g. DFT calculations. fingerprints (dict) – Dictionary with images hashs as keys and the corresponding fingerprints as values. fingerprintprimes (dict) – Dictionary with images hashs as keys and the corresponding fingerprint derivatives as values.
amp.model.send_data_to_fortran(_fmodules, energy_coefficient, force_coefficient, overfit, train_forces, num_atoms, num_images, actual_energies, actual_forces, atomic_positions, num_images_atoms, atomic_numbers, raveled_fingerprints, num_neighbors, raveled_neighborlists, raveled_fingerprintprimes, model, d)[source]

Function that sends images data to fortran code. Is used just once on each core.

## Neural Network¶

class amp.model.neuralnetwork.NeuralNetwork(hiddenlayers=(5, 5), activation='tanh', weights=None, scalings=None, fprange=None, regressor=None, mode=None, lossfunction=None, retries=0, version=None, fortran=True, checkpoints=100)[source]

Class that implements a basic feed-forward neural network.

Parameters: hiddenlayers (dict) – Dictionary of chemical element symbols and architectures of their corresponding hidden layers of the conventional neural network. Number of nodes of last layer is always one corresponding to energy. However, number of nodes of first layer is equal to three times number of atoms in the system in the case of no descriptor, and is equal to length of symmetry functions of the descriptor. Can be fed using tuples as: >>> hiddenlayers = (3, 2,)  for example, in which a neural network with two hidden layers, the first one having three nodes and the second one having two nodes is assigned (to the whole atomic system in the no descriptor case, and to each chemical element in the atom-centered mode). When setting only one hidden layer, the dictionary can be fed as: >>> hiddenlayers = (3,)  In the atom-centered mode, neural network for each species can be assigned seperately, as: >>> hiddenlayers = {"O":(3,5), "Au":(5,6)}  for example. activation (str) – Assigns the type of activation funtion. “linear” refers to linear function, “tanh” refers to tanh function, and “sigmoid” refers to sigmoid function. weights (dict) – In the case of no descriptor, keys correspond to layers and values are two dimensional arrays of network weight. In the atom-centered mode, keys correspond to chemical elements and values are dictionaries with layer keys and network weight two dimensional arrays as values. Arrays are set up to connect node i in the previous layer with node j in the current layer with indices w[i,j]. The last value for index i corresponds to bias. If weights is not given, arrays will be randomly generated. scalings (dict) – In the case of no descriptor, keys are “intercept” and “slope” and values are real numbers. In the fingerprinting scheme, keys correspond to chemical elements and values are dictionaries with “intercept” and “slope” keys and real number values. If scalings is not given, it will be randomly generated. fprange (dict) – Range of fingerprints of each chemical species. Should be fed as a dictionary of chemical species and a list of minimum and maximun, e.g.: >>> fprange={"Pd": [0.31, 0.59], "O":[0.56, 0.72]}  regressor (object) – Regressor object for finding best fit model parameters, e.g. by loss function optimization via amp.regression.Regressor. mode (str) – Can be either ‘atom-centered’ or ‘image-centered’. lossfunction (object) – Loss function object. retries (int) – If model does not converge during training, number of times to retry before giving up. version (object) – Version of this class. fortran (bool) – Can optionally shut off fortran, primarily for debugging. checkpoints (int) – Frequency with which to save parameter checkpoints upon training. E.g., 100 saves a checkpoint on each 100th training setp. Specify None for no checkpoints. Note: You can make this negative to not overwrite previous checkpoints.

:param .. note:: Dimensions of weight two dimensional arrays should be consistent: with hiddenlayers.

Raises: RuntimeError, NotImplementedError
calculate_atomic_energy(afp, index, symbol)[source]

Given input to the neural network, output (which corresponds to energy) is calculated about the specified atom. The sum of these for all atoms is the total energy (in atom-centered mode).

Parameters: afp (list) – Atomic fingerprints in the form of a list to be used as input to the neural network. index (int) – Index of the atom for which atomic energy is calculated (only used in the atom-centered mode). symbol (str) – Symbol of the atom for which atomic energy is calculated (only used in the atom-centered mode). Energy. float
calculate_dAtomicEnergy_dParameters(afp, index=None, symbol=None)[source]

Returns the derivative of energy square error with respect to variables.

Parameters: afp (list) – Atomic fingerprints in the form of a list to be used as input to the neural network. index (int) – Index of the atom for which atomic energy is calculated (only used in the atom-centered mode) symbol (str) – Symbol of the atom for which atomic energy is calculated (only used in the atom-centered mode) The value of the derivative of energy square error with respect to variables. list of float
calculate_dForce_dParameters(afp, derafp, direction, nindex=None, nsymbol=None)[source]

Returns the derivative of force square error with respect to variables.

Parameters: afp (list) – Atomic fingerprints in the form of a list to be used as input to the neural network. derafp (list) – Derivatives of atomic fingerprints in the form of a list to be used as input to the neural network. direction (int) – Direction of force. nindex (int) – Index of the neighbor atom which force is acting at. (only used in the atom-centered mode) nsymbol (str) – Symbol of the neighbor atom which force is acting at. (only used in the atom-centered mode) The value of the derivative of force square error with respect to variables. list of float
calculate_force(afp, derafp, direction, nindex=None, nsymbol=None)[source]

Given derivative of input to the neural network, derivative of output (which corresponds to forces) is calculated.

Parameters: afp (list) – Atomic fingerprints in the form of a list to be used as input to the neural network. derafp (list) – Derivatives of atomic fingerprints in the form of a list to be used as input to the neural network. direction (int) – Direction of force. nindex (int) – Index of the neighbor atom which force is acting at. (only used in the atom-centered mode) nsymbol (str) – Symbol of the neighbor atom which force is acting at. (only used in the atom-centered mode) Force. float
fit(trainingimages, descriptor, log, parallel, only_setup=False)[source]

Fit the model parameters such that the fingerprints can be used to describe the energies in trainingimages. log is the logging object. descriptor is a descriptor object, as would be in calc.descriptor.

Parameters: trainingimages (dict) – Hashed dictionary of training images. descriptor (object) – Class representing local atomic environment. log (Logger object) – Write function at which to log data. Note this must be a callable function. parallel (dict) – Parallel configuration dictionary. Takes the same form as in amp.Amp. only_setup (bool) – only_setup is primarily for debugging. It initializes all variables but skips the last line of starting the regressor.
forcetraining

Returns true if forcetraining is turned on (as determined by examining the convergence criteria in the loss function), else returns False.

get_loss(vector)[source]

Method to be called by the regression master.

Takes one and only one input, a vector of parameters. Returns one output, the value of the loss (cost) function.

Parameters: vector (list) – Parameters of the regression model in the form of a list.
get_lossprime(vector)[source]

Method to be called by the regression master.

Takes one and only one input, a vector of parameters. Returns one output, the value of the derivative of the loss function with respect to model parameters.

Parameters: vector (list) – Parameters of the regression model in the form of a list.
lossfunction

Allows the user to set a custom loss function.

For example, >>> from amp.model import LossFunction >>> lossfxn = LossFunction(energy_tol=0.0001) >>> calc.model.lossfunction = lossfxn

Parameters: lossfunction (object) – Loss function object, if at all desired by the user.
randomize(trainingimages=None, weights=True, scalings=True)[source]

Randomizes the model parameters (i.e., re-initializes them); this is typically used just before training.

Parameters: trainingimages (list) – List of ASE atoms objects that are being trained. This is only needed if scalings is True. weights (bool) – If False, do not randomize weights. scalings (bool) – If False, do not randomize scalings.
vector

Access to get or set the model parameters (weights, scaling for each network) as a single vector, useful in particular for regression.

Parameters: vector (list) – Parameters of the regression model in the form of a list.
class amp.model.neuralnetwork.NodePlot(calc)[source]

Creates plots to visualize the output of the nodes in the neural networks.

initialize with a calculator that has parameters; e.g. a trained calculator or else one in which fit has been called with the setup_only flag turned on.

Call with the ‘plot’ method, which takes as argment a list of images

plot(images, filename='nodeplot.pdf')[source]

Creates a plot of the output of each node, as a violin plot.

class amp.model.neuralnetwork.Raveler(weights, scalings)[source]

Class to ravel and unravel variable values into a single vector.

This is used for feeding into the optimizer. Feed in a list of dictionaries to initialize the shape of the transformation. Note no data is saved in the class; each time it is used it is passed either the dictionaries or vector. The dictionaries for initialization should be two levels deep.

weights, scalings are the variables to ravel and unravel

to_dicts(vector)[source]

Puts the vector back into weights and scalings dictionaries of the form initialized. vector must have same length as the output of unravel.

to_vector(weights, scalings)[source]

Puts the weights and scalings embedded dictionaries into a single vector and returns it. The dictionaries need to have the identical structure to those it was initialized with.

amp.model.neuralnetwork.calculate_dOutputs_dInputs(parameters, derafp, outputs, nsymbol)[source]
Parameters: parameters (dict) – ASE dictionary object. derafp (list) – Derivatives of atomic fingerprints in the form of a list to be used as input to the neural network. outputs (dict) – Outputs of neural network nodes. nsymbol (str) – Symbol of the atom for which atomic energy is calculated (only used in the atom-centered mode) Derivatives of outputs of neural network nodes w.r.t. inputs. dict
amp.model.neuralnetwork.calculate_nodal_outputs(parameters, afp, symbol)[source]

Given input to the neural network, output (which corresponds to energy) is calculated about the specified atom. The sum of these for all atoms is the total energy (in atom-centered mode).

Parameters: parameters (dict) – ASE dictionary object. afp (list) – Atomic fingerprints in the form of a list to be used as input to the neural network. symbol (str) – Symbol of the atom for which atomic energy is calculated (only used in the atom-centered mode) Outputs of neural network nodes dict
amp.model.neuralnetwork.calculate_ohat_D_delta(parameters, outputs, W)[source]

Calculates extra matrices ohat, D, delta needed in mathematical manipulations.

Notations are consistent with those of ‘Rojas, R. Neural Networks - A Systematic Introduction. Springer-Verlag, Berlin, first edition 1996’

Parameters: parameters (dict) – ASE dictionary object. outputs (dict) – Outputs of neural network nodes. W (dict) – The same as weight dictionary, but the last rows associated with biases are deleted in W.
amp.model.neuralnetwork.get_random_scalings(images, activation, elements=None)[source]

Generates initial scaling matrices, such that the range of activation is scaled to the range of actual energies.

images : dict
ASE atoms objects (the training set).
activation: str
Assigns the type of activation funtion. “linear” refers to linear function, “tanh” refers to tanh function, and “sigmoid” refers to sigmoid function.
elements: list of str
List of atom symbols; used in the atom-centered mode only.
Returns: scalings float
amp.model.neuralnetwork.get_random_weights(hiddenlayers, activation, len_of_fps=None, no_of_atoms=None)[source]

Generates random weight arrays from variables.

hiddenlayers: dict

Dictionary of chemical element symbols and architectures of their corresponding hidden layers of the conventional neural network. Number of nodes of last layer is always one corresponding to energy. However, number of nodes of first layer is equal to three times number of atoms in the system in the case of no descriptor, and is equal to length of symmetry functions in the atom-centered mode. Can be fed as:

>>> hiddenlayers = (3, 2,)


for example, in which a neural network with two hidden layers, the first one having three nodes and the second one having two nodes is assigned (to the whole atomic system in the case of no descriptor, and to each chemical element in the atom-centered mode). In the atom-centered mode, neural network for each species can be assigned seperately, as:

>>> hiddenlayers = {"O":(3,5), "Au":(5,6)}


for example.

activation : str
Assigns the type of activation funtion. “linear” refers to linear function, “tanh” refers to tanh function, and “sigmoid” refers to sigmoid function.
len_of_fps : dict

Length of fingerprints of each element, e.g:

>>> len_of_fps={"O": 20, "Pd":20}

no_of_atoms : int
Number of atoms in atomic systems; used only in the case of no descriptor.
Returns: weights float

## Tensorflow Neural Network¶

A work in progress, this module amp.model.tflow uses Google’s TensorFlow package to implement a neural network, which may provide GPU acceleration and other advantages.