Generic checkpoint operator. More...
#include <checkpoint.hpp>
Additional Inherited Members | |
 Public Member Functions inherited from TMBad::global::Operator< ninput, noutput > | |
| void | dependencies_updating (Args<> &args, Dependencies &dep) const |
Default implementation of OperatorPure::dependencies_updating() | |
| void * | operator_data () |
| Return operator specific dynamic information (optional) | |
| OperatorPure * | other_fuse (OperatorPure *self, OperatorPure *other) |
| How to fuse this operator (self) with another (other) | |
| void | print (print_config cfg) |
| Print this operator (optional) | |
| Static Public Attributes inherited from TMBad::global::DynamicOperator<-1, -1 > | |
| static const bool | dynamic |
| static const int | max_fuse_depth |
| Static Public Attributes inherited from TMBad::global::Operator< ninput, noutput > | |
| static const bool | add_forward_replay_copy |
| Should this operator replay it self by invoking the copy CTOR ? | |
| static const bool | add_static_identifier |
| Should this operator have a static identifier ? | |
| static const bool | allow_remap |
| Is it safe to remap the inputs of this operator? More... | |
| static const int | dependent_variable |
| Is output of this operator a dependent variable ? | |
| static const bool | dynamic |
| Does this operator require dynamic allocation ? | |
| static const bool | elimination_protected |
| Protect this operator from elimination by the tape optimizer ? | |
| static const bool | have_dependencies |
Have dependencies member been defined ? | |
| static const bool | have_eval |
Does this class have an eval member from which to define forward ? | |
| static const bool | have_forward_incr_reverse_decr |
Have forward_incr and reverse_incr members been defined ? | |
| static const bool | have_forward_mark_reverse_mark |
Have forward_mark and reverse_mark members been defined ? | |
| static const bool | have_forward_reverse |
Have forward and reverse members been defined ? | |
| static const bool | have_increment_decrement |
Have increment and decrement members been defined ? | |
| static const bool | have_input_size_output_size |
Have input_size and output_size members been defined ? | |
| static const bool | implicit_dependencies |
| Does this operator have implicit dependencies? More... | |
| static const int | independent_variable |
| Is output of this operator an independent variable ? | |
| static const bool | is_constant |
| Is this a constant operator ? | |
| static const bool | is_linear |
| Is this a linear operator ? | |
| static const int | max_fuse_depth |
| How many times can this operator be doubled ? | |
| static const int | ninput |
| Number of operator inputs. | |
| static const int | noutput |
| Number of operator outputs. | |
| static const bool | smart_pointer |
| Is this operator a 'smart pointer' (with reference counting) ? | |
| static const bool | updating |
| This operator may update existing variables ? More... | |
Detailed Description
template<class DerivativeTable>
struct TMBad::AtomOp< DerivativeTable >
Generic checkpoint operator.
This class implements checkpointing.
In short, a checkpoint operator is an atomic operator for which the derivatives are generated automatically up to any order using AD. The derivatives are stored in a derivatives table that can be shared among different instances of the operator.
There are two main use cases of interest:
- The fixed graph case where the computational graph doesn't change. The purpose of this case is to reduce memory when the same operation sequence is repeated many times. This case uses a
standard_derivative_table. - The adaptive case where the computational graph can change dynamically. The purpose of this case is to allow algorithms that use parameter dependendent branching. This case uses a
retaping_derivative_table.
Memory management
- Last operator 'alive' cleans up the shared derivatives table.
- A derivative table can always be shared among operators within the same tape. However, shared derivative tables cause an overhead because the forward pass written by one operator may be invalidated by another operator. The reverse pass must account for this by re-calulating the forward values if necessary.
- A derivative table can never be shared between different threads. A thread must have its own local (deep) copy of the table. If parallel tapes a constructed independently this is automatically the case.
Retaping
- Changed parameter inputs triggers retaping of the derivative table in the adaptive case. This happens correctly even in cases where the zero order operator is no longer present on the tape (it may have been removed by the tape optimizer).
- Note
- The fixed graph case can in principle allow for sparsity, but this is not yet implemented.
Definition at line 167 of file checkpoint.hpp.
The documentation for this struct was generated from the following file: