9jzddlZddlZddlZddlZddlmZddlZddlmcm Z ddlm Z ddl m Z ddlmZddlmZmZmZmZmZddlmZmZddlmZgd ZGd d ZGd d eZGddeZegZGddeZGddeZ GddeZ!GddeZ"dZ#GddeZ$GddeZ%GddeZ&Gdd eZ'Gd!d"eZ(Gd#d$eZ)Gd%d&eZ*Gd'd(eZ+Gd)d*eZ,Gd+d,eZ-Gd-d.eZ.Gd/d0eZ/Gd1d2eZ0y)3N)Sequence)Tensor) constraints) Distribution)_sum_rightmost broadcast_all lazy_propertytril_matrix_to_vecvec_to_tril_matrix)padsoftplus)_Number) AbsTransformAffineTransform CatTransformComposeTransformCorrCholeskyTransformCumulativeDistributionTransform ExpTransformIndependentTransformLowerCholeskyTransformPositiveDefiniteTransformPowerTransformReshapeTransformSigmoidTransformSoftplusTransform TanhTransformSoftmaxTransformStackTransformStickBreakingTransform Transformidentity_transformceZdZUdZdZej ed<ej ed<ddeddffd Z d Z e defd Z e dd Z e defd Zdd ZdZdZdZdZdZdZdZdZdZdZxZS)r!a Abstract class for invertable transformations with computable log det jacobians. They are primarily used in :class:`torch.distributions.TransformedDistribution`. Caching is useful for transforms whose inverses are either expensive or numerically unstable. Note that care must be taken with memoized values since the autograd graph may be reversed. For example while the following works with or without caching:: y = t(x) t.log_abs_det_jacobian(x, y).backward() # x will receive gradients. However the following will error when caching due to dependency reversal:: y = t(x) z = t.inv(y) grad(z.sum(), [y]) # error because z is x Derived classes should implement one or both of :meth:`_call` or :meth:`_inverse`. Derived classes that set `bijective=True` should also implement :meth:`log_abs_det_jacobian`. Args: cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. Attributes: domain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid inputs to this transform. codomain (:class:`~torch.distributions.constraints.Constraint`): The constraint representing valid outputs to this transform which are inputs to the inverse transform. bijective (bool): Whether this transform is bijective. A transform ``t`` is bijective iff ``t.inv(t(x)) == x`` and ``t(t.inv(y)) == y`` for every ``x`` in the domain and ``y`` in the codomain. Transforms that are not bijective should at least maintain the weaker pseudoinverse properties ``t(t.inv(t(x)) == t(x)`` and ``t.inv(t(t.inv(y))) == t.inv(y)``. sign (int or Tensor): For bijective univariate transforms, this should be +1 or -1 depending on whether transform is monotone increasing or decreasing. Fdomaincodomain cache_sizereturnNcz||_d|_|dk(rn|dk(rd|_n tdt|y)Nr)NNzcache_size must be 0 or 1) _cache_size_inv _cached_x_y ValueErrorsuper__init__)selfr& __class__s ^/media/conek/DATA/Code/OCR/venv/lib/python3.12/site-packages/torch/distributions/transforms.pyr/zTransform.__init__asB%=A ?  1_)D 89 9 cD|jj}d|d<|S)Nr+)__dict__copy)r0states r2 __getstate__zTransform.__getstate__ls" ""$f  r3c|jj|jjk(r|jjStd)Nz:Please use either .domain.event_dim or .codomain.event_dim)r$ event_dimr%r-r0s r2r:zTransform.event_dimqs: ;; DMM$;$; ;;;(( (UVVr3cd}|j|j}|%t|}tj||_|S)z{ Returns the inverse :class:`Transform` of this transform. This should satisfy ``t.inv.inv is t``. N)r+_InverseTransformweakrefref)r0invs r2r@z Transform.invwsB  99 ))+C ;#D)C C(DI r3ct)z Returns the sign of the determinant of the Jacobian, if applicable. In general this only makes sense for bijective transforms. NotImplementedErrorr;s r2signzTransform.signs "!r3c|j|k(r|St|jtjurt||St t|d)Nr&z.with_cache is not implemented)r*typer/r!rCr0r&s r2 with_cachezTransform.with_cachesU   z )K :  )"4"4 44:4 4!T$ZL0N"OPPr3c ||uSNr0others r2__eq__zTransform.__eq__s u}r3c&|j| SrK)rOrMs r2__ne__zTransform.__ne__s;;u%%%r3c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z2 Computes the transform `x => y`. r)r*_callr,)r0xx_oldy_oldys r2__call__zTransform.__call__sY   q ::a= '' u :L JJqMa4r3c|jdk(r|j|S|j\}}||ur|S|j|}||f|_|S)z1 Inverts the transform `y => x`. r)r*_inverser,)r0rWrUrVrTs r2 _inv_callzTransform._inv_calls[   q ==# #'' u :L MM! a4r3ct)zD Abstract method to compute forward transformation. rBr0rTs r2rSzTransform._call "!r3ct)zD Abstract method to compute inverse transformation. rBr0rWs r2rZzTransform._inverser^r3ct)zU Computes the log det jacobian `log |dy/dx|` given input and output. rBr0rTrWs r2log_abs_det_jacobianzTransform.log_abs_det_jacobianr^r3c4|jjdzS)Nz())r1__name__r;s r2__repr__zTransform.__repr__s~~&&--r3c|S)z{ Infers the shape of the forward computation, given the input shape. Defaults to preserving shape. rLr0shapes r2 forward_shapezTransform.forward_shape  r3c|S)z} Infers the shapes of the inverse computation, given the output shape. Defaults to preserving shape. rLrhs r2 inverse_shapezTransform.inverse_shaperkr3r)r'r!r))re __module__ __qualname____doc__ bijectiver Constraint__annotations__intr/r8propertyr:r@rDrIrOrQrXr[rSrZrcrfrjrm __classcell__r1s@r2r!r!0s*XI  " ""$$$ 3 t  W3WW   "c""Q&  " " " .r3r!ceZdZdZdeddffd ZejddZejdd Z e de fd Z e de fd Ze defd Zdd ZdZdZdZdZdZdZxZS)r=z| Inverts a single :class:`Transform`. This class is private; please instead use the ``Transform.inv`` property. transformr'NcHt||j||_yNrF)r.r/r*r+)r0r{r1s r2r/z_InverseTransform.__init__s  I$9$9:( r3F is_discretec\|j td|jjSN_inv must not be None)r+AssertionErrorr%r;s r2r$z_InverseTransform.domains* 99  !89 9yy!!!r3c\|j td|jjSr)r+rr$r;s r2r%z_InverseTransform.codomains* 99  !89 9yyr3c\|j td|jjSr)r+rrsr;s r2rsz_InverseTransform.bijectives( 99  !89 9yy"""r3c\|j td|jjSr)r+rrDr;s r2rDz_InverseTransform.signs& 99  !89 9yy~~r3c|jSrK)r+r;s r2r@z_InverseTransform.invs yyr3cz|j td|jj|jSr)r+rr@rIrHs r2rIz_InverseTransform.with_caches3 99  !89 9xx"":.222r3ct|tsy|j td|j|jk(S)NFr) isinstancer=r+rrMs r2rOz_InverseTransform.__eq__s9%!23 99  !89 9yyEJJ&&r3c`|jjdt|jdS)N())r1rereprr+r;s r2rfz_InverseTransform.__repr__ s)..))*!DO+>r3cf|j td|jj|Sr)r+rr[r]s r2rXz_InverseTransform.__call__s- 99  !89 9yy""1%%r3cj|j td|jj|| Sr)r+rrcrbs r2rcz&_InverseTransform.log_abs_det_jacobians2 99  !89 9 ..q!444r3c8|jj|SrK)r+rmrhs r2rjz_InverseTransform.forward_shapeyy&&u--r3c8|jj|SrK)r+rjrhs r2rmz_InverseTransform.inverse_shaperr3ro)rerprqrrr!r/rdependent_propertyr$r%rwboolrsrvrDr@rIrOrfrXrcrjrmrxrys@r2r=r=s ))))$[##6"7" $[##6 7 #4## c Y3 '?& 5 ..r3r=c eZdZdZddeededdffd ZdZe jd d Z e jd d Z e defd Ze defd ZedefdZddZdZdZdZdZdZxZS)rab Composes multiple transforms in a chain. The transforms being composed are responsible for caching. Args: parts (list of :class:`Transform`): A list of transforms to compose. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. partsr&r'Nc~|r|Dcgc]}|j|}}t| |||_ycc}wr})rIr.r/r)r0rr&partr1s r2r/zComposeTransform.__init__,s? =BCTT__Z0CEC J/ Ds:cVt|tsy|j|jk(SNF)rrrrMs r2rOzComposeTransform.__eq__2s#%!12zzU[[((r3Fr~cB|jstjS|jdj}|jdjj }t |jD]R}||jj |jj z z }t||jj }T||j krtd|d|j ||j kDr#tj|||j z }|S)Nr event_dim z must be >= domain.event_dim ) rrrealr$r%r:reversedmaxr independent)r0r$r:rs r2r$zComposeTransform.domain7szz## #A%%JJrN++55 TZZ( >D ..1H1HH HIIt{{'<'<=I > v'' ' YK'DVEUEUDVW  v'' ' ,,VYAQAQ5QRF r3c0|jstjS|jdj}|jdjj }|jD]R}||jj |jj z z }t ||jj }T||j krtd|d|j ||j kDr#tj|||j z }|S)Nrrrz must be >= codomain.event_dim ) rrrr%r$r:rrr)r0r%r:rs r2r%zComposeTransform.codomainJszz## #::b>**JJqM((22 JJ @D 004;;3H3HH HIIt}}'>'>?I @ x)) ) YK'FxGYGYFZ[  x)) )"..xXEWEW9WXHr3c:td|jDS)Nc34K|]}|jywrKrs).0ps r2 z-ComposeTransform.bijective.._s311;;3)allrr;s r2rszComposeTransform.bijective]s3 333r3cJd}|jD]}||jz}|SNr))rrD)r0rDrs r2rDzComposeTransform.signas, !A!&&=D ! r3c$d}|j|j}|jtt|jDcgc]}|jc}}t j ||_t j ||_|Scc}wrK)r+rrrr@r>r?)r0r@rs r2r@zComposeTransform.invhsn 99 ))+C ;"8DJJ3G#HaAEE#HIC C(DI{{4(CH $IsB cR|j|k(r|St|j|Sr})r*rrrHs r2rIzComposeTransform.with_cachess&   z )K zBBr3c8|jD] }||} |SrK)r)r0rTrs r2rXzComposeTransform.__call__xs#JJ DQA r3c p|jstj|S|g}|jddD]}|j||d|j|g}|jj }t |j|dd|ddD]x\}}}|jt|j||||jj z ||jj |jj z z }ztjtj|S)Nrr))rtorch zeros_likeappendr$r:ziprrcr% functoolsreduceoperatoradd)r0rTrWxsrtermsr:s r2rcz%ComposeTransform.log_abs_det_jacobian}szz##A& &SJJsO $D IId2b6l # $ ! KK)) djj"Sb'2ab6: IJD!Q LL--a3YAVAV5V  004;;3H3HH HI  I e44r3cJ|jD]}|j|}|SrK)rrjr0rirs r2rjzComposeTransform.forward_shapes*JJ .D&&u-E . r3c\t|jD]}|j|}|SrK)rrrmrs r2rmzComposeTransform.inverse_shapes/TZZ( .D&&u-E . r3c|jjdz}|dj|jDcgc]}|j c}z }|dz }|Scc}w)Nz( z, z ))r1rejoinrrf)r0 fmt_stringrs r2rfzComposeTransform.__repr__sT^^,,y8 innDJJ%Gqajjl%GHH e &HsA rnro)rerprqrrlistr!rvr/rOrrr$r%r rrsrDrwr@rIrXrcrjrmrfrxrys@r2rr!sd9o3t ) $[##67"$[##67"4444c YC  5*  r3rc eZdZdZ ddedededdffd ZddZejd d Z ejd d Z e de fd Ze defdZdZdZdZdZdZdZxZS)ra Wrapper around another transform to treat ``reinterpreted_batch_ndims``-many extra of the right most dimensions as dependent. This has no effect on the forward or backward transforms, but does sum out ``reinterpreted_batch_ndims``-many of the rightmost dimensions in :meth:`log_abs_det_jacobian`. Args: base_transform (:class:`Transform`): A base transform. reinterpreted_batch_ndims (int): The number of extra rightmost dimensions to treat as dependent. base_transformreinterpreted_batch_ndimsr&r'Nc`t|||j||_||_yr})r.r/rIrr)r0rrr&r1s r2r/zIndependentTransform.__init__s0 J/,77 C)B&r3ch|j|k(r|St|j|j|Sr})r*rrrrHs r2rIzIndependentTransform.with_caches5   z )K#   !?!?J  r3Fr~cjtj|jj|jSrK)rrrr$rr;s r2r$zIndependentTransform.domains.&&    & &(F(F  r3cjtj|jj|jSrK)rrrr%rr;s r2r%zIndependentTransform.codomains.&&    ( ($*H*H  r3c.|jjSrK)rrsr;s r2rszIndependentTransform.bijectives"",,,r3c.|jjSrK)rrDr;s r2rDzIndependentTransform.signs""'''r3c|j|jjkr td|j |SNToo few dimensions on input)dimr$r:r-rr]s r2rSzIndependentTransform._calls7 557T[[** *:; ;""1%%r3c|j|jjkr td|jj |Sr)rr%r:r-rr@r`s r2rZzIndependentTransform._inverses= 557T]],, ,:; ;""&&q))r3cj|jj||}t||j}|SrK)rrcrr)r0rTrWresults r2rcz)IndependentTransform.log_abs_det_jacobians1$$99!Q?(F(FG r3cz|jjdt|jd|jdS)Nrz, r)r1rerrrr;s r2rfzIndependentTransform.__repr__s:..))*!D1D1D,E+FbIgIgHhhijjr3c8|jj|SrK)rrjrhs r2rjz"IndependentTransform.forward_shape""0077r3c8|jj|SrK)rrmrhs r2rmz"IndependentTransform.inverse_shaperr3rnro)rerprqrrr!rvr/rIrrr$r%rwrrsrDrSrZrcrfrjrmrxrys@r2rrs " C!C$'C C  C $[##6 7 $[##6 7 -4--(c((& *  k88r3rc eZdZdZdZ ddej dej deddffd Ze jd Z e jd Z dd Z d Zd ZdZdZdZxZS)ra Unit Jacobian transform to reshape the rightmost part of a tensor. Note that ``in_shape`` and ``out_shape`` must have the same number of elements, just as for :meth:`torch.Tensor.reshape`. Arguments: in_shape (torch.Size): The input event shape. out_shape (torch.Size): The output event shape. cache_size (int): Size of cache. If zero, no caching is done. If one, the latest single value is cached. Only 0 and 1 are supported. (Default 0.) Tin_shape out_shaper&r'Nctj||_tj||_|jj |jj k7r t dt ||y)Nz6in_shape, out_shape have different numbers of elementsrF)rSizerrnumelr-r.r/)r0rrr&r1s r2r/zReshapeTransform.__init__sc  8, I. ==   DNN$8$8$: :UV V J/r3cptjtjt|jSrK)rrrlenrr;s r2r$zReshapeTransform.domains&&&{'7'7T]]9KLLr3cptjtjt|jSrK)rrrrrr;s r2r%zReshapeTransform.codomains&&&{'7'7T^^9LMMr3ch|j|k(r|St|j|j|Sr})r*rrrrHs r2rIzReshapeTransform.with_caches,   z )K t~~*UUr3c|jd|jt|jz }|j ||j zSrK)rirrrreshaper)r0rT batch_shapes r2rSzReshapeTransform._calls?gg<#dmm*< <= yyt~~566r3c|jd|jt|jz }|j ||j zSrK)rirrrrr)r0rWrs r2rZzReshapeTransform._inverse#s?gg=#dnn*= => yyt}}455r3c|jd|jt|jz }|j |SrK)rirrr new_zeros)r0rTrWrs r2rcz%ReshapeTransform.log_abs_det_jacobian's6gg<#dmm*< <= {{;''r3c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSNrzShape mismatch: expected z but got )rrr-rr0ricuts r2rjzReshapeTransform.forward_shape+s u:DMM* *:; ;%j3t}}-- ;$-- '+E#$K= $--Q Tc{T^^++r3c t|t|jkr tdt|t|jz }||d|jk7rtd||dd|j|d||jzSr)rrr-rrs r2rmzReshapeTransform.inverse_shape5s u:DNN+ +:; ;%j3t~~.. ;$.. (+E#$K= $..AQR Tc{T]]**r3rnro)rerprqrrrsrrrvr/rrr$r%rIrSrZrcrjrmrxrys@r2rrs I  0** 0:: 0 0  0##M$M##N$NV 76(,+r3rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz8 Transform via the mapping :math:`y = \exp(x)`. Tr)c"t|tSrK)rrrMs r2rOzExpTransform.__eq__J%..r3c"|jSrK)expr]s r2rSzExpTransform._callM uuwr3c"|jSrKlogr`s r2rZzExpTransform._inversePrr3c|SrKrLrbs r2rcz!ExpTransform.log_abs_det_jacobianSr3Nrerprqrrrrr$positiver%rsrDrOrSrZrcrLr3r2rr@s=  F##HI D/r3rceZdZdZej Zej ZdZdde de ddffd Z ddZ e de fd Zd Zd Zd Zd ZdZdZxZS)rzD Transform via the mapping :math:`y = x^{\text{exponent}}`. Texponentr&r'NcJt||t|\|_yr})r.r/rr)r0rr&r1s r2r/zPowerTransform.__init__`s" J/(2r3cR|j|k(r|St|j|Sr})r*rrrHs r2rIzPowerTransform.with_cacheds&   z )Kdmm CCr3c6|jjSrK)rrDr;s r2rDzPowerTransform.signis}}!!##r3ct|tsy|jj|jj j Sr)rrreqritemrMs r2rOzPowerTransform.__eq__ms:%0}}/335::<|jd|jz Srrr`s r2rZzPowerTransform._inverseusuuQ&''r3c^|j|z|z jjSrK)rabsrrbs r2rcz#PowerTransform.log_abs_det_jacobianxs( !A%**,0022r3cXtj|t|jddSNrirLrbroadcast_shapesgetattrrrhs r2rjzPowerTransform.forward_shape{"%%eWT]]GR-PQQr3cXtj|t|jddSrr rhs r2rmzPowerTransform.inverse_shape~r r3rnro)rerprqrrrrr$r%rsrrvr/rIr rDrOrSrZrcrjrmrxrys@r2rrWs ! !F##HI33S33D $c$$= $(3RRr3rctj|j}tjtj||j d|j z SN?minr)rfinfodtypeclampsigmoidtinyeps)rTrs r2_clipped_sigmoidrs< KK E ;;u}}Q'UZZS599_ MMr3c`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rzg Transform via the mapping :math:`y = \frac{1}{1 + \exp(-x)}` and :math:`x = \text{logit}(y)`. Tr)c"t|tSrK)rrrMs r2rOzSigmoidTransform.__eq__%!122r3ct|SrK)rr]s r2rSzSigmoidTransform._calls ""r3ctj|j}|j|jd|j z }|j | jz Sr)rrrrrrrlog1p)r0rWrs r2rZzSigmoidTransform._inversesK AGG$ GG eiiG 8uuw1"%%r3c\tj|  tj|z SrK)Fr rbs r2rcz%SigmoidTransform.log_abs_det_jacobians! A2A..r3N)rerprqrrrrr$ unit_intervalr%rsrDrOrSrZrcrLr3r2rrs=  F((HI D3#& /r3rc`eZdZdZej ZejZdZ dZ dZ dZ dZ dZy) rz Transform via the mapping :math:`\text{Softplus}(x) = \log(1 + \exp(x))`. The implementation reverts to the linear function when :math:`x > 20`. Tr)c"t|tSrK)rrrMs r2rOzSoftplusTransform.__eq__s%!233r3ct|SrKr r]s r2rSzSoftplusTransform._calls {r3cb| jjj|zSrK)expm1negrr`s r2rZzSoftplusTransform._inverses'zz|!%%'!++r3ct|  SrKr&rbs r2rcz&SoftplusTransform.log_abs_det_jacobians! }r3NrrLr3r2rrs=   F##HI D4,r3rcneZdZdZej ZejddZdZ dZ dZ dZ dZ d Zy ) ra Transform via the mapping :math:`y = \tanh(x)`. It is equivalent to .. code-block:: python ComposeTransform( [ AffineTransform(0.0, 2.0), SigmoidTransform(), AffineTransform(-1.0, 2.0), ] ) However this might not be numerically stable, thus it is recommended to use `TanhTransform` instead. Note that one should use `cache_size=1` when it comes to `NaN/Inf` values. grTr)c"t|tSrK)rrrMs r2rOzTanhTransform.__eq__s%//r3c"|jSrK)tanhr]s r2rSzTanhTransform._calls vvxr3c,tj|SrK)ratanhr`s r2rZzTanhTransform._inverses{{1~r3cVdtjd|z td|zz zS)N@g)mathrr rbs r2rcz"TanhTransform.log_abs_det_jacobians*dhhsma'(4!8*<<==r3N)rerprqrrrrr$intervalr%rsrDrOrSrZrcrLr3r2rrsF,  F#{##D#.HI D0 >r3rcReZdZdZej ZejZdZ dZ dZ y)rz*Transform via the mapping :math:`y = |x|`.c"t|tSrK)rrrMs r2rOzAbsTransform.__eq__rr3c"|jSrK)rr]s r2rSzAbsTransform._callrr3c|SrKrLr`s r2rZzAbsTransform._inverserr3N) rerprqrrrrr$rr%rOrSrZrLr3r2rrs*5   F##H/r3rc eZdZdZdZ ddeezdeezdededdf fd Ze defd Z e jd d Z e jd dZddZdZe deezfdZdZdZdZdZdZxZS)ra Transform via the pointwise affine mapping :math:`y = \text{loc} + \text{scale} \times x`. Args: loc (Tensor or float): Location parameter. scale (Tensor or float): Scale parameter. event_dim (int): Optional size of `event_shape`. This should be zero for univariate random variables, 1 for distributions over vectors, 2 for distributions over matrices, etc. Tlocscaler:r&r'NcPt||||_||_||_yr})r.r/r:r; _event_dim)r0r:r;r:r&r1s r2r/zAffineTransform.__init__s* J/ #r3c|jSrK)r=r;s r2r:zAffineTransform.event_dims r3Fr~c|jdk(rtjStjtj|jSNrr:rrrr;s r2r$zAffineTransform.domain9 >>Q ## #&&{'7'7HHr3c|jdk(rtjStjtj|jSr@rAr;s r2r%zAffineTransform.codomainrBr3c~|j|k(r|St|j|j|j|Sr})r*rr:r;r:rHs r2rIzAffineTransform.with_cache!s7   z )K HHdjj$..Z  r3c8t|tsyt|jtr4t|jtr|j|jk7r7y|j|jk(j j syt|j tr5t|j tr|j |j k7ryy|j |j k(j j syy)NFT)rrr:rrrr;rMs r2rOzAffineTransform.__eq__(s%1 dhh (Z 7-Kxx599$HH )..0557 djj' *z%++w/OzzU[[( JJ%++-22499;r3ct|jtr6t|jdkDrdSt|jdkrdSdS|jj S)Nrr)r)rr;rfloatrDr;s r2rDzAffineTransform.sign<sR djj' *djj)A-1 Utzz9JQ9N2 UTU Uzz  r3c:|j|j|zzSrKr:r;r]s r2rSzAffineTransform._callBsxx$**q.((r3c:||jz |jz SrKrIr`s r2rZzAffineTransform._inverseEsDHH  **r3c|j}|j}t|tr3t j |t jt|}n#t j|j}|jrQ|jd|j dz}|j|jd}|d|j }|j|S)N)rr)rir;rrr full_liker3rrr:sizeviewsumexpand)r0rTrWrir;r result_sizes r2rcz$AffineTransform.log_abs_det_jacobianHs  eW %__QU(<=FYYu%))+F >> ++-(94>>/:UBK[[-11"5F+T^^O,E}}U##r3c tj|t|jddt|jddSrrr r r:r;rhs r2rjzAffineTransform.forward_shapeU7%% 7488Wb174::wPR3S  r3c tj|t|jddt|jddSrrSrhs r2rmzAffineTransform.inverse_shapeZrTr3rrro)rerprqrrrsrrGrvr/rwr:rrr$r%rIrOrDrSrZrcrjrmrxrys@r2rrs I  $ e^ $~ $ $  $  $3$[##6I7I $[##6I7I  (!fsl!! )+ $  r3rcdeZdZdZej ZejZdZ dZ dZ d dZ dZ dZy) ra Transforms an unconstrained real vector :math:`x` with length :math:`D*(D-1)/2` into the Cholesky factor of a D-dimension correlation matrix. This Cholesky factor is a lower triangular matrix with positive diagonals and unit Euclidean norm for each row. The transform is processed as follows: 1. First we convert x into a lower triangular matrix in row order. 2. For each row :math:`X_i` of the lower triangular part, we apply a *signed* version of class :class:`StickBreakingTransform` to transform :math:`X_i` into a unit Euclidean length vector using the following steps: - Scales into the interval :math:`(-1, 1)` domain: :math:`r_i = \tanh(X_i)`. - Transforms into an unsigned domain: :math:`z_i = r_i^2`. - Applies :math:`s_i = StickBreakingTransform(z_i)`. - Transforms back into signed domain: :math:`y_i = sign(r_i) * \sqrt{s_i}`. Tctj|}tj|jj}|j d|zd|z }t |d}|dz}d|z jjd}|tj|jd|j|jz}|t|dddfddgd z}|S) Nrr)rdiag)rdevice.rvalue) rr.rrrrr sqrtcumprodeyerir\r )r0rTrrzz1m_cumprod_sqrtrWs r2rSzCorrCholeskyTransform._callus JJqMkk!''"&& GGSa#gG . qr * qDE<<>11"5  !''"+QWWQXXF F $S#2#X.Aa@ @r3c,dtj||zdz }t|dddfddgd}t|d}t|d}||j z }|j |j j z dz }|S) Nr)rr.rr]rYr[)rcumsumr r r_rr))r0rWy_cumsumy_cumsum_shiftedy_vec y_cumsum_vectrTs r2rZzCorrCholeskyTransform._inversesu||AEr22xSbS1Aq6C"12.)*:D \'') ) WWY (A -r3Ncd||zjdz }t|d}d|jjdz}d|t d|zzt jdz jdz}||zS)Nr)rrfrY?r2)rgr rrOr r3)r0rTrW intermediates y1m_cumsumy1m_cumsum_trilstick_breaking_logdet tanh_logdets r2rcz*CorrCholeskyTransform.log_abs_det_jacobians !a%B// -ZbA #&;&;&=&A&A"&E EAa 00488C=@EE"EMM ${22r3ct|dkr td|d}tdd|zzdzdz}||dz zdz|k7r td|dd||fzS)Nr)rrg?r[roz.Input is not a flattened lower-diagonal number)rr-round)r0riNDs r2rjz#CorrCholeskyTransform.forward_shapest u:>:; ; "I 4!a%:; ; 9b !23 3 "I QK1 SbzQD  r3rK)rerprqrrr real_vectorr$ corr_choleskyr%rsrSrZrcrjrmrLr3r2rr`s=  $ $F((HI   3#!r3rc^eZdZdZej ZejZdZ dZ dZ dZ dZ y)ra< Transform from unconstrained space to the simplex via :math:`y = \exp(x)` then normalizing. This is not bijective and cannot be used for HMC. However this acts mostly coordinate-wise (except for the final normalization), and thus is appropriate for coordinate-wise optimization algorithms. c"t|tSrK)rrrMs r2rOzSoftmaxTransform.__eq__rr3c||}||jdddz j}||jddz S)NrTr)rrrO)r0rTlogprobsprobss r2rSzSoftmaxTransform._calls@HLLT2155::<uyyT***r3c&|}|jSrKr)r0rWrs r2rZzSoftmaxTransform._inversesyy{r3c8t|dkr td|SNr)rrzrhs r2rjzSoftmaxTransform.forward_shape u:>:; ; r3c8t|dkr td|Srrzrhs r2rmzSoftmaxTransform.inverse_shaperr3N)rerprqrrrr{r$simplexr%rOrSrZrjrmrLr3r2rrs8 $ $F""H3+  r3rcheZdZdZej ZejZdZ dZ dZ dZ dZ dZdZy ) r a Transform from unconstrained space to the simplex of one additional dimension via a stick-breaking process. This transform arises as an iterated sigmoid transform in a stick-breaking construction of the `Dirichlet` distribution: the first logit is transformed via sigmoid to the first probability and the probability of everything else, and then the process recurses. This is bijective and appropriate for use in HMC; however it mixes coordinates together and is less appropriate for optimization. Tc"t|tSrK)rr rMs r2rOzStickBreakingTransform.__eq__%!788r3c(|jddz|j|jdjdz }t||j z }d|z j d}t |ddgdt |ddgdz}|S)Nrr)rr])rinew_onesrgrrr`r )r0rToffsetrc z_cumprodrWs r2rSzStickBreakingTransform._callsq1::aggbk#:#A#A"#EE Q- .UOOB' Aq6 #c)aV1&E Er3c|dddf}|jd|j|jdjdz }d|jdz }tj|tj |j j}|j|jz |jz}|S)N.rr))r) rirrgrrrrrr)r0rWy_croprsfrTs r2rZzStickBreakingTransform._inverses38qzz&,,r*:;BB2FF r" "[[QWW!5!:!: ; JJL2668 #fjjl 2r3c,|jddz|j|jdjdz }||jz }| t j |z|dddfjzj d}|S)Nrr).)rirrgrr! logsigmoidrO)r0rTrWrdetJs r2rcz+StickBreakingTransform.log_abs_det_jacobiansq1::aggbk#:#A#A"#EE  Q\\!_$qcrc{'88==bA r3cRt|dkr td|dd|ddzfzSNr)rrrzrhs r2rjz$StickBreakingTransform.forward_shape5 u:>:; ;SbzU2Y],,,r3cRt|dkr td|dd|ddz fzSrrzrhs r2rmz$StickBreakingTransform.inverse_shape rr3N)rerprqrrrr{r$rr%rsrOrSrZrcrjrmrLr3r2r r sB  $ $F""HI9- -r3r cteZdZdZej ej dZejZ dZ dZ dZ y)rz Transform from unconstrained matrices to lower-triangular matrices with nonnegative diagonal entries. This is useful for parameterizing positive definite matrices in terms of their Cholesky factorization. r[c"t|tSrK)rrrMs r2rOzLowerCholeskyTransform.__eq__rr3c|jd|jddjjzSNrrn)dim1dim2)trildiagonalr diag_embedr]s r2rSzLowerCholeskyTransform._call4vvbzAJJBRJ8<<>IIKKKr3c|jd|jddjjzSr)rrrrr`s r2rZzLowerCholeskyTransform._inverse!rr3N) rerprqrrrrrr$lower_choleskyr%rOrSrZrLr3r2rrs?%[ $ $[%5%5q 9F))H9LLr3rcteZdZdZej ej dZejZ dZ dZ dZ y)rzN Transform from unconstrained matrices to positive-definite matrices. r[c"t|tSrK)rrrMs r2rOz PositiveDefiniteTransform.__eq__-s%!:;;r3c@t|}||jzSrK)rmTr]s r2rSzPositiveDefiniteTransform._call0s $ " $Q '144xr3crtjj|}tj |SrK)rlinalgcholeskyrr@r`s r2rZz"PositiveDefiniteTransform._inverse4s* LL ! !! $%'++A..r3N) rerprqrrrrrr$positive_definiter%rOrSrZrLr3r2rr%s=%[ $ $[%5%5q 9F,,H</r3rc eZdZUdZeeed< ddeededeedzdeddf fd Z e defd Z e defd Z dd Z d ZdZdZedefdZej*dZej*dZxZS)ra Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim`, of length `lengths[dim]`, in a way compatible with :func:`torch.cat`. Example:: x0 = torch.cat([torch.range(1, 10), torch.range(1, 10)], dim=0) x = torch.cat([x0, x0], dim=0) t0 = CatTransform([ExpTransform(), identity_transform], dim=0, lengths=[10, 10]) t = CatTransform([t0, t0], dim=0, lengths=[20, 20]) y = t(x) transformsNtseqrlengthsr&r'ctd|Ds td|r|Dcgc]}|j|}}t||t ||_|dgt|j z}t ||_t|jt|j k7r8tdt|jdt|j d||_ ycc}w)Nc3<K|]}t|tywrKrr!rrls r2rz(CatTransform.__init__..Q::a+:0All elements of tseq must be Transform instancesrFr)z lengths (z) must match transforms (r) rrrIr.r/rrrrr)r0rrrr&rlr1s r2r/zCatTransform.__init__Js:T:: !ST T 6:;ALL,;D; J/t* ?cC00GG} t|| DOO 4 4 C -..GDOOH\G]]^_ .b811;;8r)rrr;s r2r:zCatTransform.event_dim`8888r3c,t|jSrK)rOrr;s r2lengthzCatTransform.lengthds4<<  r3c||j|k(r|St|j|j|j|SrK)r*rrrrrHs r2rIzCatTransform.with_cachehs2   z )KDOOTXXt||ZPPr3c|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d}t |j |j D]>\}}|j|j||}|j||||z}@tj||jS) Ndim  out of range for tensor with dimensionsx.size() =  must equal length rrf) rrrMrrrrnarrowrrcat)r0rTyslicesstarttransrxslices r2rSzCatTransform._callmsDHH.quuw. txxj >quuwi{S  66$(( t{{ * $((4txx(8'99LT[[MZ  $,,? #ME6XXdhhv6F NN5= )FNE #yydhh//r3c|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d}t |j |j D]G\}}|j|j||}|j|j|||z}Itj||jS) Nrrry.size(rrrrf) rrrMrrrrrrr@rr)r0rWxslicesrrryslices r2rZzCatTransform._inverse~sDHH.quuw. txxj >quuwi{S  66$(( t{{ * $((4txx(8'99LT[[MZ  $,,? #ME6XXdhhv6F NN599V, -FNE #yydhh//r3cf|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|j|j |jcxkr|jks,ntd|jd|jd|j|j|jk7rAtd|jd|j|jd|jg}d }t |j |j D]\}}|j|j||}|j|j||}|j||} |j|jkr#t| |j|jz } |j| ||z}|j} | d k\r| |jz } | |jz} | d krtj|| St|S) Nr out of range for x with rrrr out of range for y with rrrf)rrrMrrrrrrcr:rrrrrO) r0rTrW logdetjacsrrrrr logdetjacrs r2rcz!CatTransform.log_abs_det_jacobians=DHH.quuw. txxj 9!%%'+N  66$(( t{{ * $((4txx(8'99LT[[MZ DHH.quuw. txxj 9!%%'+N  66$(( t{{ * $((4txx(8'99LT[[MZ   $,,? #ME6XXdhhv6FXXdhhv6F2266BI/*9dnnu6VW   i (FNE #hh !8-CDNN" 799ZS1 1z? "r3c:td|jDS)Nc34K|]}|jywrKrrs r2rz)CatTransform.bijective..rrrrr;s r2rszCatTransform.bijectiverr3ctj|jDcgc]}|jc}|j|j Scc}wrK)rrrr$rrr0rls r2r$zCatTransform.domains:# /!QXX /4<<  /Actj|jDcgc]}|jc}|j|j Scc}wrK)rrrr%rrrs r2r%zCatTransform.codomains:!% 1AQZZ 1488T\\  1r)rNrro)rerprqrrrr!rurrvr/r r:rrIrSrZrcrwrrsrrr$r%rxrys@r2rr9s Y (, y!#%     ,9399!!!Q 0"0"##J9499## $ ## $ r3rc eZdZUdZeeed< ddeedededdffd Z dd Z d Z d Z d Z d ZedefdZej&dZej&dZxZS)raW Transform functor that applies a sequence of transforms `tseq` component-wise to each submatrix at `dim` in a way compatible with :func:`torch.stack`. Example:: x = torch.stack([torch.range(1, 10), torch.range(1, 10)], dim=1) t = StackTransform([ExpTransform(), identity_transform], dim=1) y = t(x) rrrr&r'Nctd|Ds td|r|Dcgc]}|j|}}t||t ||_||_ycc}w)Nc3<K|]}t|tywrKrrs r2rz*StackTransform.__init__..rrrrF)rrrIr.r/rrr)r0rrr&rlr1s r2r/zStackTransform.__init__se:T:: !ST T 6:;ALL,;D; J/t*quuwi{S  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  QA *MFE NN5= ) *{{711r3c |j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|jg}t |j ||jD]%\}}|j|j|'tj||jS)Nrrrrrrrf) rrrMrrrrrr@rr)r0rWrrrs r2rZzStackTransform._inversesDHH.quuw. txxj >quuwi{S  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  QA .MFE NN599V, - .{{711r3c r|j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|j|j |jcxkr|jks,ntd|jd|jd|j|jt|jk7rJtd|jd|j|jdt|jg}|j |}|j |}t |||jD]'\}}}|j|j||)tj||j S) Nrrrrrrrrrf) rrrMrrrrrrcrr) r0rTrWrrrrrrs r2rcz#StackTransform.log_abs_det_jacobiansDHH.quuw. txxj 9!%%'+N  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl DHH.quuw. txxj 9!%%'+N  66$(( s4??3 3 $((4txx(8'99UVYZ^ZiZiVjUkl  ++a.++a.%('4??%K J !FFE   e88H I J{{:48844r3c:td|jDS)Nc34K|]}|jywrKrrs r2rz+StackTransform.bijective..rrrr;s r2rszStackTransform.bijectiverr3ctj|jDcgc]}|jc}|jScc}wrK)rrrr$rrs r2r$zStackTransform.domain!s1  DOO!Dq!((!DdhhOO!DActj|jDcgc]}|jc}|jScc}wrK)rrrr%rrs r2r%zStackTransform.codomain&s1  doo!F!**!FQQ!FrrVro)rerprqrrrr!rurrvr/rIrrSrZrcrwrrsrrr$r%rxrys@r2rrs YJK Y' .1 CF   E H 2 2509499##P$P##R$Rr3rceZdZdZdZej ZdZdde de ddffd Z e dejdzfd Zd Zd Zd Zdd ZxZS)raA Transform via the cumulative distribution function of a probability distribution. Args: distribution (Distribution): Distribution whose cumulative distribution function to use for the transformation. Example:: # Construct a Gaussian copula from a multivariate normal. base_dist = MultivariateNormal( loc=torch.zeros(2), scale_tril=LKJCholesky(2).sample(), ) transform = CumulativeDistributionTransform(Normal(0, 1)) copula = TransformedDistribution(base_dist, [transform]) Tr) distributionr&r'Nc4t||||_yr})r.r/r)r0rr&r1s r2r/z(CumulativeDistributionTransform.__init__Cs J/(r3c.|jjSrK)rsupportr;s r2r$z&CumulativeDistributionTransform.domainGs  (((r3c8|jj|SrK)rcdfr]s r2rSz%CumulativeDistributionTransform._callKs  $$Q''r3c8|jj|SrK)ricdfr`s r2rZz(CumulativeDistributionTransform._inverseNs  %%a((r3c8|jj|SrK)rlog_probrbs r2rcz4CumulativeDistributionTransform.log_abs_det_jacobianQs  ))!,,r3cR|j|k(r|St|j|Sr})r*rrrHs r2rIz*CumulativeDistributionTransform.with_cacheTs(   z )K.t/@/@ZXXr3rnro)rerprqrrrsrr"r%rDrrvr/rwrtr$rSrZrcrIrxrys@r2rr,st$I((H D)\)s)4)) ..5))()-Yr3r)1rr3rr>collections.abcrrtorch.nn.functionalnn functionalr!rtorch.distributionsr torch.distributions.distributionrtorch.distributions.utilsrrr r r r r torch.typesr__all__r!r=rr"rrrrrrrrrrrrr rrrrrrLr3r2rsg $ +9. 0ffRE. E.PyD&b)K89K8\I+yI+X9.(RY(RVN /y/2 0*>I*>Z 9  h ih VP!IP!f!y!H5-Y5-pLYL,/ /(K 9K \bRYbRJ+Yi+Yr3