3.0 Release Note¶

New Features¶

• Added PaddlePaddle APIs to expand PaddlePaddle’s functionality. These include paddle.nn.FeatureAlphaDropout, paddle.cartesian_prod, paddle.distributed.to_distributed, paddle.pi, etc. #64881, #65605, #70757, #71030, #69946, #70021, #69613, #68123, #70032

• Introduce new Tensor class methods and attributes, along with corresponding unit tests, to enhance the usability of Tensor. #68334, #68681, #69132, #69270, #69256, #69197, #69231, #69222, #69257, #69301, #69361, #69348, #69464, #69542, #69667, #69563, #69796, #69477, #69779, #69724, #69835, #69781, #69982, #69913, #70026, #70013, #69539, #69736, #69841, #70277, #69580, #69599, #69693, #69848, #69751, #70556, #70591, #69673, #70647, #68192, #68511, #68833, #69406, #69480, #69463, #69632, #69473, #68694, #69534, #69820, #70121

API Function Enhancement¶

• Enhanced the functionality of 43 APIs, making existing APIs easier to use and facilitating code conversion. This includes but is not limited to adding API parameters, expanding the data types supported by APIs, and correcting existing unreasonable designs. #65105, #65103, #62975, #64436, #63346, #68079, #67878, #68432, #68677, #69012, #69385, #65032, #64977, #67071, #67298, #66687, #65946, #66170, #66929, #67994, #67947, #68033, #68046, #68294, #68214, #68281, #68390, #68772, #69451, #69252, #69529, #69750, #69827, #69099, #68594, #70090, #70228, #70166, #70389, #70790, #71029, #71283, #71342

• PaddlePaddle Python API fully supports type hints. All parameters and return values of Python API have been annotated with type hints for ease of development and use. #65209, #65201, #65190, #65082, #65226, #65076, #65238, #65236, #65247, #65249, #65244, #65272, #65191, #65290, #65255, #65292, #65300, #65301, #65332, #65323, #65326, #65273, #65317, #65354, #65283, #65372, #65337, #65085, #65382, #65381, #65378, #65274, #65380, #65386, #65351, #65284, #65366, #65308, #65375, #65376, #65464, #65197, #65455, #65457, #65487, #65486, #65547, #65504, #65460, #65183, #65454, #65559, #65560, #65570, #65569, #65566, #65620, #65568, #65567, #65660, #65645, #65600, #65532, #65765, #65767, #65770, #65768, #65771, #65772, #65774, #65769, #65773, #65766, #65776, #65775, #65755, #65779, #65777, #65823, #65807, #65821, #65819, #65810, #65808, #65824, #65553, #65818, #65812, #65803, #65865, #65870, #65866, #65844, #65845, #65853, #65874, #65871, #65809, #65867, #65822, #65872, #65873, #65869, #65868, #65849, #65875, #65876, #65843, #65727, #65587, #66006, #66005, #65785, #65784, #65811, #65919, #65838, #65852, #65847, #66014, #65805, #66009, #66012, #65633, #66011, #66010, #66013, #66015, #66016, #66030, #66028, #66029, #66054, #66040, #65993, #66058, #66280, #66037, #66057, #66077, #66051, #65912, #66090, #66189, #66127, #66277, #66119, #66270, #66305, #66306, #66279, #66276, #66295, #66301, #66473, #66384, #66505, #66328, #66394, #66392, #66432, #66575, #66572, #66656, #66475, #66654, #66616, #66694, #66686, #66766, #66749, #66760, #66803, #66770, #66693, #66771, #66792, #66862, #66867, #66684, #66966, #66793, #66987, #66985, #66989, #66639, #66994, #66986, #66993, #67002, #66996, #67001, #66864, #67031, #67089, #67143, #67179, #67178, #67284, #67104, #67079, #67132, #67147, #67204, #67112, #67233, #67366, #67067, #67391, #67428, #67197, #67047, #66890, #67159, #67439, #67555, #67448, #67556, #67469, #67558, #67405, #67644, #67624, #67679, #67677, #67785, #67767, #65319, #65277, #67673, #65557, #67527, #66965, #65905, #65657, #66357, #68163

• Optimized the error messages of many PaddlePaddle APIs, making the errors more understandable. #67148, #67154, #67546, #67335, #67255, #67099, #67074, #67073, #66957, #67063, #67575, #67608, #67634, #67325, #67429, #67401, #66881, #68492, #67695, #69833, #70398

Bug Fixes¶

• Fixed a bug in paddle.nn.functional.max_unpool1d when the input output_size is a tuple. #65910

• Fixed the issue where paddle.base.core.eager.Tensor did not support paddle::DataType. #66765

• Fixed the issue where an error occurred during BF16 training when the pir switch was turned on. #66833

• Fixed the issue of bias in the linear layer during parallel processing in the pipeline. #67212

• Fixed the error issue when using loss for judgment in parallel pipeline. #66980

• Fixed the error issue when using paddle.Tensor.item in parallel pipeline. #67441

• Fixed bugs in paddle.einsum in specific scenarios. #67588

• Fixed the error issue of paddle.nn.SyncBatchNorm during gradient computation. #67559

• Fixed the issue mentioned in issue #69992. #70017

• Fixed the issue where paddle.arange produced incorrect results when dealing with large integers. #70188

• Fixed the issue where paddle.max and paddle.min propagated incorrectly when there were nan values in the input. #70049

• Fixed issues with APIs such as paddle.linalg.svd and paddle.linalg.any when handling 0-size Tensor. #70235, #70489, #70047, #70103, #70127, #70098, #70077, #70130, #70254, #70125, #70342, #70369, #71094, #71089, #71185, #70537, #70481

• Fixed some issues with type hint annotations and documentation issues. #65429, #65496, #65461, #65542, #65575, #65545, #65609, #65644, #65700, #65697, #65719, #65639, #65742, #65891, #65877, #65895, #66007, #66679, #66680, #66676, #66677, #66884, #67288, #67302, #66978, #67295, #67520, #67421, #67529, #67536, #67618, #67661, #67698, #67800, #67933, #67893, #68108, #67927, #68322, #68341, #68415, #68372, #68559, #68598, #68708, #68780, #68992, #68989, #68895, #69014, #69139, #68996, #69090, #68922, #69333, #69141, #69609, #69652, #69715, #69716, #69934, #70253, #70297, #70252, #70468, #70102, #70546, #70616, #70582, #70635, #70499, #70755, #70935, #71133, #71172, #71238, #71230, #71394

Document optimization¶

• Enhanced several API documents to make them easier to read and understand. #67772, #69895, #65904, #66480, #66974, #67100, #66991, #67287, #67841, #68206, #68305, #68462, #67061, #66503, #68856, #68866, #68768, #69215, #69449, #69396, #69498, #69413, #69404, #69729, #69749, #69266, #69989, #70209, #70128, #70143, #69874, #70242, #70145, #70813, #71046

Bug Fixes¶

• Fixed accuracy issues caused by parameter configuration. #65814

• Fixed bugs related to save/load. #65268, #65359, #65373, #65314, #65446, #65476, #66891, #66931, #65978, #67654, #67906, #68723, #71452, #71457, #67819, #68120, #68300, #68315, #68743, #68744, #69585, #71165, #71400

• Skip/fix failed unit tests in PIR mode, including scenarios such as Windows and XPU. #65690, #65759, #65730, #65760, #65833, #65834, #65856, #65886, #65899, #65932, #65998, #65953, #65997, #66061, #66111, #66137, #66073, #66203, #66227, #65744, #66234, #67487, #67561, #67584, #67742, #69832, #65885, #66709, #66734, #66959, #67399, #67389, #67230, #67403, #67619, #67662, #67902, #67382, #67430, #67517, #67533, #67573, #67468, #67640, #67667, #67716, #68386, #67234, #67266, #67362, #67631, #68081

• Fixed bugs related to dynamic graphs. #65619, #69163, #68862, #68164, #69867

• Fixed bugs related to control flow. #65722, #70181

• Fixed kernel operation-related bugs, including issues with operation positions and null pointers. #66334, #67931, #70353

• Fixed Amp-related bugs. #66778, #67582, #67704, #68655

• Fixed CINN-related bugs. #69577, #71101, #71387, #71401

• Fixed the bug related to the transition from dynamic to static. #67617, #67936, #68938, #68734, #69010, #69408, #69461, #69699, #69774, #69803, #69853, #70510, #70830, #70904, #70913, #71040, #71048, #71106, #71201, #71216, #71223, #71296, #71385, #71505, #66934, #71096, #71144, #71430, #71437, #71473, #71412, #65648, #67853, #66543, #68229, #70846, #67532

• Fixed other bugs, including issues related to backpropagation gradient calculation, memory copying, and executor errors. #65493, #65678, #65673, #65794, #66358, #66875, #67339, #67465, #67754, #67835, #67892, #67967, #67952, #68036, #68063, #68128, #68151, #68140, #68167, #68200, #68325, #68376, #68539, #68530, #68637, #68639, #68688, #68751, #68806, #68810, #68779, #68811, #68844, #68790, #68870, #68960, #68999, #69036, #69188, #69234, #69375, #69399, #69538, #69603, #69633, #69765, #69768, #69821, #70091, #70123, #70147, #70201, #70198, #69815, #70420, #70377, #70552, #70545, #70595, #70836, #70771, #70922, #70969, #70926, #71117, #71151, #71194, #71234, #71339, #71445, #66350, #66533, #66622, #67721, #67700, #69207, #69615, #69785, #67805

Function optimization¶

• Support save/load. #65296, #65671, #66231, #66185, #66722, #66863, #67057, #68101, #68628, #66359, #68481

• Optimize the compilation process of custom operators. #67615, #67659

• Support for composite operators. #69121, #69144, #70204, #71098, #71335

• Support for CINN compiler execution. #69589, #70115

• Support for custom devices. #70909, #71294, #71362, #71010, #71036, #70637, #71085

• Execution support for other scenarios. #65050, #65664, #65741, #65786, #65499, #66441, #67668, #68199, #69088, #70199, #70308, #70709, #70937, #71066, #71079, #71121, #71136, #71205

New Features¶

• SOT adapts to Python 3.13 bytecode, supporting static graph conversion (SOT mode) under Python 3.13. #68071, #69126, #69131, #69196, #69232, #69253, #69267, #69412, #69431, #69432, #69436, #69557, #69567, #69700, #69707, #69735, #69738, #69744, #69753, #69887, #69920, #69950, #70319, #70927

• Support for custom devices. #68061, #68836, #70366, #70549

• Adapted PIR forward execution. #65335

• Support save/load. #67910

• Adapted to pylayer. #70335

• Adapt lazy_init. #67379, #67467

• Optimize the logic under PIR. #67961

• Support for other scenarios. #68344, #70071, #70291, #70752, #70812, #71033

Changes unrelated to ordinary users¶

• Optimize SOT debugging for experience and improve development efficiency. #67560, #69072, #69837, #70134, #70387, #70740, #71118, #71268, #71275, #71458, #71460

• Other changes unrelated to user usage. #65393, #65795, #65799, #65911, #65977, #66982, #67563, #68761, #68909, #69130, #69233, #69956, #71142

Security Issues¶

• Introduced approval rules for IR (Intermediate Representation) save/load operations to enhance security and governance during model serialization. #65737

Others¶

• Sparse API migration. #66139, #66319, #66866

• PIR function extension. #67966, #69909

• Migrate file locations. #66477, #66824, #67592

• Log addition. #68382, #70506

• Enable PIR by default. #68278

• Header file organization. #68422, #68471

• Compilation optimization. #67831, #67821, #68717

• Manage related tests with guards. #67816, #67827, #67989

• Fixed spelling errors. #70784, #70787

• Check for CUDA errors. #70399

Developer¶

• Fix issues in dynamic-to-static conversion. Improve overall graph conversion success rate and optimize inference export experience. #65291, #66153, #66379, #66557, #67021, #67482, #67495, #67981, #68030, #68078, #68328, #68442, #68679, #68850, #68892, #68991, #69043, #69097, #69210, #69295, #69428, #69518, #69642, #69940, #70118, #70169, #70218, #70287, #70412, #71099, #71156, #71193, #71336, #71463, #71476, #71503

• Inplace strategy upgrade. #65491

• Control flow related development. #67251

• Add environment variables. #68467

• Support sparse operator operations. #67111

• Other execution support development, including logic optimization, version adaptation, and adding unit tests. #69241, #69806, #70768, #66829, #67110, #67442, #67041, #67452, #69061, #69307, #68669, #69829, #70003, #70443, #70364, #71495

Performance optimization¶

• Optimize dynamic shape handling in static graph conversion, reducing graph construction iterations and compilation time. #65235, #65477, #65517, #65882, #66346, #66746, #67786, #67876, #68113, #68302, #68337, #68616, #69354, #70009, #70877

• End-to-end performance optimization for SOT, minimizing subgraph fragmentation, reducing scheduling overhead, and improving static training efficiency. #67591, #67746, #67823, #67890, #67921, #68031, #68153, #68729, #69249, #69263, #69300, #69313, #69325, #69353, #69411, #69506, #69672, #69746, #69834, #69836, #69852, #69975, #70151, #70293, #70405, #70851, #71039, #71254, #71295, #71298, #71346, #71377, #71407

• Optimize the performance of dynamic shape scenarios. #68491, #68629

• Accelerate the execution speed of PIR executor. #69513

• Optimize PIR saving and loading performance. #69683

• Optimize for device. #69676

• Clean up redundant input and output information. #66278

Discontinued Features¶

• Remove outdated test cases. #66269, #66690, #67505, #67464, #68400, #68178, #68194

• Clean up obsolete flags and configurations. #69124, #69176, #69274, #68384

• Eliminate old APIs. #66032, #67303

• Cleaned up PIR redundancy strategy and single test. #66366, #70534, #68444, #70599, #68801, #66303, #67854, #70795

• Discard the related unit tests and APIs for dynamic-to-static conversion. #66421, #68251, #68252, #68253, #68254, #68409, #70569, #71279

• Discard the related unit tests for automatic parallelism. #67857, #67862, #67995, #68012, #68013, #67798

New Features¶

1. New hardware backend support: Added support for two new backends, HIP and SYCL. (#65146, #65329, #69554, #71204, #65438, #66476, #66620, #67813)

2. Added support for manual setting of numerical ranges, equality constraints, and other information for symbol dimensions in reasoning scenarios. (#67628, #67384)

Function optimization¶

1. Optimize the printing of error messages to enhance the development and debugging experience. (#67738, #68769, #71076)

2. Support the Welford algorithm, which can simultaneously ensure the performance and accuracy of the BatchNorm-related operator Kenrel. (#71184, #71057)

Performance optimization¶

1. New backend optimization strategies such as GridReduce, Loop merging, Transpose tuning, and automatic vectorization have been added, significantly enhancing Kernel performance across various dimensional spaces and under different hardware configurations in all scenarios. (#67236, #68897, #69409, #65336, #66419, #68338, #68364, #71087, #68019, #68122, #65187, #66742, #67083, #68667, #68750, #69376, #69350, #69740, #68918, #70092, #69607, #69794, #70258, #70547, #70581, #70649, #69732, #70786, #70942, #71014, #71263, #71249, #71340, #71301, [#71380](https://github.com

2. Optimize operator fusion strategies, upgrading various strategies including horizontal fusion, multi-downstream fusion, Reshape alignment fusion, etc., to further enhance the fusion capabilities of operators and improve end-to-end optimization performance. (#66034, #67829, #68171, #69478, #69691, #70665, #71103, #70873)

3. The simplification capability of backend subscript expressions has been upgraded, supporting the simplification of complex expressions with dynamic and static dimensions, significantly reducing the subscript computation overhead in the generated backend Kernel. (#68011, #68617, #68624, #68685, #68220, #68720, #68753, #68986, #68987, #69071, #69164, #69282, #69522, #69857, #70208, #70355, #70427, #70450, #68737, #70500, #70953, #70933, #71026, #70456, #70257, #70461, #70142, #71018, #71278)

4. A new automatic Re-Compute mechanism for reverse computation graphs has been added, which can effectively reduce model training memory usage and improve performance. (#69342, #70255, #68241, #69954, #70832)

5. Optimize the backend Host and Device code compilation process to reduce compilation time and improve the processing performance of branches in the Broadcast scenario. (#65669, #65916, #66109, #65611, #65990, #66088, #66207, #66537, #66768, #70685, #71410, #66062)

6. Improved and upgraded the mechanisms for symbol derivation, simplification, and caching in dynamic dimensions, added symbol derivation interface implementations for all conventional operators (580+), and provided more constraint information for Kernel compilation.(#65343、#66582、#65500、#65591、#66637、#68208、#68056、#68015、#68096、#68236、#68973、#68967、#69133、#68550、#68882、#69005、#69911、#70376、#71153、#66644、#66650、#66642、#66729、#66838、#66762、#66580、#66612、#66625、#66643、#66837、#66946、#67018、#67049、#66956、#67008、#66930、#66877、#66896、#67120、#67117、#67098、#67136、#67294、#67327、#66827、#67201、#66892、#67377、#66619、#67037、#67412、#67394、#67374、#67418、#67348、#67337、#67390、#67407、#67491、#67422、#67461、#67458、#67486、#67490、#67462、#67364、#67435、#67665、#67426、#67507、#67730、#67776、#67806、#67803、#67788、#67705、#67814、#67858、#67751、#67875、#67663、#67434、#67818、#68180、#68547、#68548、#68670、#68964、#68929、#68907、#68917、#68984、#68644、#69167、#68975、#68947、#68978、#68980、#68979、#69329、#69055、#69331、#69414、#69335、#69017、#69344、#69069、#69698、#69919、#69964、#70337、#70282、#70741、#70818、#71031、#70541、#66609、#66889、#66633、#66735、#66935、#66627、#66730、#67210、#67115、#67275、#67472、#67577、#67328、#67566、#67451、#68098、#68225、#68177、#68102、#67951、#67957、#68235、#68447、#68446、#68183、#68318、#68385、#67635、#65623、#65956、#66063、#65992、#65880、#66343、#65889、#66606、#66618、#66737、#66607、#66579、#66732、#66849、#66400、#66952、#66570、#66967、#66595、#67121、#67206、#67444、#67494、#67499、#67267、#67567、#67455、#67161、#67581、#67539、#67625、#67690、#67454、#67731、#67734、#67735、#67607、#67413、#67387、#67882、#67864、#67503、#67861、#67888、#67884、#67826、#68044、#67851、#68276、#69888、#70093、#70436、#70914、#71222)

7. Optimized some front-end passes to enhance the robustness of the front-end processing flow and improve the performance of computationally intensive subgraphs. (#65142, #67466, #69228, #70994, #71226, #71297, #71443)

8. Designed new backend IR basic components and related Pass interfaces to provide a more concise and efficient way of developing optimization strategies. Through automatic pruning strategies, it can effectively reduce the traversal overhead of backend IR. (#70485, #70765, #71042, #70952, #69454, #70361, #70334, #70406, #70191, #70462, #70548, #70592, #70437, #70619, #70543, #69611, #70739, #70533, #70696, #70498, #70829, #71111, #70883)

Bug fixes¶

1. Fix some bugs in the derivation and implementation logic of operator symbols. (#65185, #65231, #65266, #65951, #67142, #67286, #65958, #65955, #66470, #66764, #66036, #66662, #66741, #66745, #66807, #66791, #66859, #66880, #66962)

2. Fixed bugs in the lowering of some special operators to the compiler. (#68698, #68699, #68691, #68948, #70144, #70895)

3. Fixed the issue of errors reported in some scenarios when integrating operators. (#67038, #67400, #67655, #67723, #68029, #68042, #68888, #69250, #69937, #70924)

4. Fix the correctness issue of the backend when handling extreme values, and improve the robustness of the compiler. (#68327)

5. Fixed implementation logic bugs in the backend Schedule and post-processing tuning process, resolving errors and performance issues in some cases. (#68605, #68937, #68587, #69060, #69608, #71471, #71068)

6. Resolved the issue of randomness in the operator fusion process. (#69547, #70931)

New Features¶

• Added the paddle.distributed.parallel interface to support configuring common parallel strategies outside of model networking, simplifying the distributed development process. #69004, #69033, #69077, #69136, #69169, #69212, #69217, #69283, #69288, #69326, #69365, #69384, #69426, #69443, #69462, #69492, #69628, #69677, #69697, #69776, #69896, #70138, #70182, #70539, #71116, #71210

• For pure text sparse scenarios, it supports MoE expert parallelism, implements an expert parallelism to mesh partitioning conversion mechanism, and supports automatic invocation of all2all communication. #66462, #66750, #68004, #68053, #68187, #68477, #69098, #69262, #69296, #70715, #71292, #71320

• To meet the needs of users in extreme manual optimization scenarios for managing segmentation status and communication operations, and to address the issue of being unable to use tensor segmentation syntax in some non-SPMD scenarios, we have added the LocalLayer interface to support a hybrid network of automatic and manual parallelism. #70519, #70525, #70600, #71232, #71264, #71373

• To enable users to run automatic parallel programs using domestic hardware, we have completed the adaptation for Kunlun chips, and support for other chips is also underway. #70997, #71126, #71229, #71289, #71425, #71500

• For situations where the data dimension cannot be divided evenly by the device dimension, non-balanced splitting derivation and splitting transformation are supported. #66103, #67756, #69265, #70072

• The shard_dataloader function has been upgraded to support setting the gradient accumulation step count through batch_sampler, and also supports scenarios with multiple model inputs. #65325, #70659

• Upgrades have been made to the parameter saving and loading functions, supporting asynchronous storage of parameters, mutual loading of master_weight between dynamic and static graphs, as well as parameter version control and offload functions. #66858, #67427, #70105, #70639

• To meet users’ needs for converting dynamic networking involving PyLayer to static, support has been added for PyLayer in static graph mode, allowing distributed tensors to be run within PyLayer. #67326, #68190, #69089, #70831

• To address the issue of incorrect dynamic-to-static conversion caused by inconsistency between the data stream input format and the input_spec actually required by the model for dynamic-to-static conversion, the dynamic-to-static conversion interface supports a user-defined input_spec feature, allowing users to input the required input_spec on their own. #69183

• For hybrid parallel scenarios, the gradient clipping strategy has been adapted and supported. #65259, #65928, #69287, #69760, #71421

• For scenarios where the number of model layers is not divisible by the number of devices, a non-balanced pipeline parallel strategy is supported, allowing users to split different numbers of network layers at different pipeline stages. #69728, #70164, #70230

• Added set_mesh and get_mesh interfaces to enable users to easily set and retrieve the global mesh. #69999

• Added automatic and manual parallelism accuracy alignment switches to facilitate the conversion of existing manual parallelism models to automatic parallelism and verify the accuracy of the results. #67681

Functional improvements¶

Improve and optimize the derivation rules for operator slicing

• Added derivation rules for operators add_n, split, and softmax_grad. #65606, #69439

• Added operator splitting derivation rules for assign and embedding_grad. #67457

• Added clip operator slicing derivation rule. #70632

• Added derivation rules for the dist_stack and gather_nd operators. #65426

• Added the derivation rule for dropout operator segmentation. #70216

• Added slicing derivation rule for fused_dropout_add operator. #67722

• Added fast_ln custom operator segmentation derivation rule. #68148

• Added greater_equal and less_equal operator slicing derivation rules. #68868

• Added greater_than and less_than operator slicing derivation rules. #68133

• Added if operator segmentation derivation rule. #69357

• Added slicing derivation rules for operators logical_and, logical_not, logical_or, and logical_xor. #67840

• Added logsumexp operator slicing derivation rule. #67840

• Added non_zero operator slicing derivation rule. #67996

• Added pad operator slicing derivation rule. #68304

• Added the derivation rule for operator segmentation of p_norm. #68317

• Added the derivation rule for the scatter_nd operator’s slicing. #67980

• Added sigmoid operator segmentation derivation rule. #71092

Static graph automatic parallel architecture based on PIR upgrade

• Upgrades to Automatic Mixed Precision (AMP) training. #65089, #65892, #66418, #66674, #68545

• Upgrade of recalculation strategy. #69681, #70064

• Upgrades to the parameter slicing parallel strategy. #63542, #67748, #68288, #68314, #69059, #71167

• Upgrading the pipeline parallelism strategy. #66810, #67174, #67522, #68141, #68742, #68962, #69052, #69201, #69244, #69578, #69584, #69654, #69799, #69894, #70360, #70615

• Gradient accumulation strategy upgrade. #66641, #67254, #67907, #68391, #68460, #68472, #68664, #68727, #69171, #69805

• Operator fusion strategy upgrade. #68087, #68207, #68383, #68623, #68650, #68736, #69103, #70889

• The tensor_fusion optimization strategy has been upgraded. #66130, #68475, #69243, #69560, #69823, #70195, #70309, #70363, #70869

• Tensor parallel optimization strategy upgrade. #68182, #68389

• Upgrade of custom operator segmentation derivation mechanism. #67614

• Upgrades to the parameter saving and loading mechanism. #66416, #67045, #67369, #68203

• Optimize computation graph compilation time. #68796

Bug fixes¶

• Fixed bugs in the segmentation derivation mechanism and the segmentation derivation rules for several operators. #65702, #65835, #66098, #66955, #67052, #67059, #67101, #67283, #67729, #67996, #68413, #68455, #68533, #68976, #68977, #69027, #69203, #69223, #69862, #69991, #70100, #70624, #71024, #71152, #71214, #71253, #71388

• Fixed several bugs in the segmentation conversion mechanism. #65060, #65820, #67630, #67809, #68115, #68468, #70023

• Fixed the bug of incorrect derivation of shard_degree in parameter slice parallelism. #68781, #69214

• Fixed issues in scenarios such as inconsistent results between dynamic and static graphs in shard_dataloader, slicing dict-type data, and custom sampler scenarios. #65262, #66096, #66882, #69620

• Fixed the bug where the recompute setting with use_reentrant=false was incompatible with parameter slicing. #65188

• Fixed bugs in the parameter loading and saving functions. #66266, #69764

• Fixed bugs in operators such as Conv2D, fill_constant, flash_attn_grad, reduce_scatter, if, tuple_push, and tuple_pop. #67587, #68008, #68586, #68589, #69519, #70207

• Fixed bugs in communication operators such as reduce_scatter, p_send, and p_recv. #67386, #71433

• Fixed bugs related to tensor type promotion. #66541, #68342

• Fixed the bug where automatic allocation of GPU memory occurred when converting uninitialized distributed tensors to NumPy arrays on some cards. #66361

• Fixed the bug that triggered data copying when calling to_tensor on non-segmented tensors. #67169

• Fixed the bug related to the segmentation of the scaler parameter. #68289

• Fixed the accuracy issue of enable_delay_scale_loss. #68525

• Fixed the hang issue caused by different creation orders of communication groups. #68847

• Fixed the bug of incorrect op_role setting in static graph scenarios. #67850, #67986, #68156

• Fixed the bug where the output variable of the random number operator could not be sliced in static graphs. #67589, #67750, #68067

• Fixed the bug where the graph cache mechanism failed in static graphs. #68488

• Fixed the bug of index out-of-bounds in paddle.distributed.to_distributed. #70174

• Fixed a bug in the pipeline parallel visualization tool. #71386

New Features¶

• Support the splitting of combinatory operators. #65148, #65007, #65482, #65006, #65692, #65961, #65968, #65967, #66510, #66795, #66835, #67151, #67342, #67481, #67502, #67606, #67757, #67775, #67891, #67790, #67965, #67968, #68168, #68125, #68228, #68295, #68353, #68357, #68827, #68834, #69239, #68817, #69108, #69373, #69372, #68829, #69684, #68818, #68835, #69838, #69998, #69675, #70367, #70080, #71352, #66450, #67593, #67988, #68346, #68399, #68319, #68485, #68961, #68575

• PIR supports Pylayer. #69674, #70375

• Support for XPU-related operator computations. #65684, #65976, #68497

• PIR supports sparse operators. #62663, #67885, #67976, #68261, #68326

• Support manual Recompute. #65879

• Implement the kernel and register the operator. #63130

• Support for Custom Op. #68824, #68748

• Added dynamic graph second-order inverse composition for acos. #70409

• Support initialization and computation of 0-size tensors. #70504

Bug Fixes¶

• Fixed bugs related to composite operators. #70250, #67170, #71218, #69095, #70189

• Fixed XPU-related bugs. #65149, #70845

• Fixed shape-related bugs. #68722, #70210, #70492

• Fixed save/load-related bugs. #69153

• Fixed bugs related to types. #65721, #65859

• Fixing issues during the invocation and execution of other operators, including type matching, type inference, parameter type support, etc,. #65360, #65024, #66308, #67085, #67285, #67076, #67547, #68007, #68527, #68549, #68543, #68604, #68741, #68859, #69025, #69065, #69405, #69688, #69912, #70177, #70517, #70596, #70788, #70870, #71332, #71454, #71442, #71499, #67459, #68470, #70206

Others¶

• Optimize code style. #68536

• Fix spelling errors. #67456, #66673, #68702, #68735, #68718, #70700, #70682, #70670, #70241, #69626, #70051, #67764, #68872, #70055, #67954, #67404, #69273, #66981, #68145, #69148, #69145, #69168, #68940, #70344

• Modify the interface documentation. #69378

• Replaced operator and parameter naming under the fluid operator system. #69345, #69382, #69484, #69444

Discarded¶

• xshape output exit. #66769, #67009, #67152, #67172, #67355, #67373, #66089

• Remove the obsolete operators, their kernels, related unit tests, and related calling codes under the fluid system. #67370, #67088, #67324, #67666, #68058, #68311, #68358, #68312, #68355, #67528, #68316, #68356, #68397, #68441, #68417, #68567, #68583, #68649, #68331, #68730, #69754, #69445, #69921, #70268, #69446, #69544, #70272, #69745, #70300, #70388, #70421, #70302, #70445, #69275, #69081, #70588, #67778, #67953, #68093, #68092, #67684, #69665, #67915, #67917, #68403, #68404, #68969, #68953, #68954, #68942, #68950, #69381, #69380, #69448, #69680, #69775, #69812, #69840, #69828, #69742, #69923, #69922, #69904, #70002, #70054, #70052, #70053, #70713, #70718, #70718, #70717

• Remove deprecated flags. #70727, #70726

• Remove the deprecated API of combination operators. #69873, #69309

Developer-related¶

• Support for composition operators, including adapter operators, adding flags, test cases, etc. #67725, #65252, #67590, #68076, #66711, #68813, #68928, #69054, #69156, #69255, #69460, #70270

• Add unit tests for operators. #68272, #68490

• Added operator API aliases for PaddleCustomDevice. #69526

• Define the position of the shift operator to ensure it only supports dynamic graphs. #69289

• Annotate only forward computation operators. #68580

• Change the inverse operator of the view operation to reuse the forward operator, thereby supporting the need for higher-order differentiation in scientific computing scenarios. #71086

• Migrate operator file location/modify function namespace/modify function parameter names, etc. #66393, #67066, #67012, #67243, #67367, #67760, #67242, #67189, #67899, #67687, #68035, #67682, #68464, #68469, #67900, #68563, #68562, #68564, #68479, #68588, #68726, #68719, #68767, #68557, #68671, #68786, #67948, #64999, #68581, #68361, #68656, #68396, #68059, #68785, #68665, #68869, #67626, #68921, #69268, #69271, #69306, #69302, #69341, #69364, #69343, #69383, #69415, #69437, #69494, #69541, #69543, #69540, #69569, #69568, #69621, #69622, #69701, #69702, #69704, #69743, #69780, #69814, #69822, #69893, #69967, #69976, #70011, #70015, #70007, #70010, #70346, #70414, #69951, #70299, #70441, #70435, #68420, #70671, #70705, #68540, #70211, #67489, #66927, #66942, #66848, #66796, #67036, #67244, #67299, #67171, #67293, #67208, #67408, #67523, #67689, #67694, #67797, #67894, #65969, #65939, #67928, #68097, #66744, #68496, #66943, #68773, #69272

• Move test file locations. #67564, #68266, #68634

• Pre-modification related to xshape output exit. #67543, #67572

Improvement¶

• Supported more data types. #69143

• Update xpu interface. #69800

• Improved operator printing functionality. #69916

• Upgraded the normalize operation to support more scenarios. #70152

• Extended group_norm to handle cases where the rank is greater than 5. #68774

• Improved the usage of backward_blacklist. #69356

Performance improvement¶

• Optimized the performance of the where_double_grad operator. #70404

• Change “for range” to “slice” to speed up the execution of grad. #69938

New Features¶

• Enhanced support for fp8 type. #64735, #64955

• Enhanced support for XPU. #65362, #65304, #68451

• Enhanced support for DCU. #65398, #65857, #66423

• Expand the capabilities of oneDNN. #66000, #66474, #66568

• Rename parameters and support more complex masks. #65409

• Support for flash-attention. #68968

• Support OpenVINO CPU high-performance inference. #69122

Functional improvements¶

• Enhance PIR pass to achieve better fusion. #65540

• Enhanced OneDNN functionality. #65971, #70430, #70630, #70871

• Improve the performance of FlashMask. #68109

• Optimize kernel performance. #69660, #69596

• Combinatorial operator optimization. #69515, #69616

Bug Fixes¶

• Fixed bugs related to PIR, CINN, SOT, OneDNN, etc. #68951, #69553, #69682, #67741, #69346, #69401, #68903

• Fixed bugs related to composite operators. #69479, #69487, #67176

• Fixed the issue with the FP8 data type on the CPU. #65539

• Remove unnecessary overhead for creating events in computational flow. #67315

• Fixed performance issues. #68378

• Fixed issues related to types. #69720

• Fixed other issues. #70019, #70008, #70645, #71209, #68152, #69907, #71207

Performance optimization¶

• Optimizations related to the CINN compiler. #69455, #70284, #67576, #68946, #68615

• Optimizations related to oneDNN. #68784, #68716, #67554

• Memory-related optimizations. #68660, #69930, #68174, #68660, #70359

• Kernel computation-related optimizations. #65507, #68541, #71479, #71403

• XPU-related optimizations. #67051

• Other optimizations include pass optimization of the inference process, dynamic shape optimization in automatic parallelism, and FlashAttention computation optimization. #68394, #68696, #68759, #68791, #69390, #69961, #69939, #70455, #70663, #71290

Others¶

• Modify function namespaces. #66818, #67023, #67114, #67217, #67524, #67796, #67881 Upgrade OneDNN. #69917

• Modify the pass level. #69524

• Optimizations related to memory read and write. #65804, #66923

• Optimize the GetValueName-related signatures. #66363, #66559, #66738

Discarded¶

• Remove obsolete files and functions. #67514, #67811, #67911

New Features¶

• Support Paddle-TensorRT based on PaddlePaddle’s new generation of intermediate representation (PIR)

• Development of core basic execution mechanism functions and code optimization. #64995, #67054, #67660, #67755, #70762,

• Development of operator Marker and Converter. #67753，#67956，#68084，#67974，#68395，#68216，#68529，#68608， #68663，#68757，#68614，#68783，#68775，#68839，#68686，#68840，#68941，#69015，#69038，#69117，#69208，#69315，#69261，#68878，#69705，#69706，#70170，#70267，#70429，#69330，#70507，#70535，#70667，#70816，#70826，#70955，#71028，#71013，#71157，#71231，#69199，#68956，#66658，#66811，#67519，#67877，#68090，#69086，#68787，#68778，#69318，#69995，#70325，#70817，#70879，#70875，#71041，#68876

• Support for Generic Plugin execution function. #66634, #70251

• Low-precision (FP16, INT8) function support. #69597, #71127,

• Auxiliary functions such as the single test system and pass usage support have been improved #67525, #68034, #71281, #71235, #67568, #70139, #70529

• Large model inference optimization

• Added fused_moe function support (basic support/non-standard TopK/Hopper architecture) #66084, #67425, #67732

• Support for mixed precision computation (GQA mixed precision/BF16 registration) #65078, #67769

• Added inference optimization features (dynamic graph inference/support for 128K long sequences) #65962, #70088

• Added implementation of quantization inference operator (FP8 W8A8 computation/weight-only int4 quantization) #65441, #64094

Feature-complete¶

• The functional mechanism of Inference is well-established under PIR

• The executor supports loading .json models #65223

• Support controllable PIR mode switch-on/off #65596

• Improved reasoning mechanism of large models

• Optimized gemm algorithm search (cublaslt global search/offline caching) #65597, #66132

• Enhance type system compatibility (PD_VISIT_FLOATING_AND_HALF_TYPES) #71022

• Optimized attention mechanism (support for multiple blocks of MMHA/XPU) #67211, #68104

Performance optimization¶

• OneDNN has been upgraded to version 3.6, resulting in a general improvement in model inference performance on GNR/EMR devices #69386

• Operator performance optimization (layer_norm/top_p_sampling) #65711

• Model loading acceleration (regular/PIR model) #69110, #70219

Bug fixes¶

• Fixed issues related to Predictor when saving/loading PIR models. #65180, #65019, #65714, #69619, #67570, #65595, #69200

• Fixed execution issues of reasoning unit tests in scenarios such as PIR and multiple hardware configurations. #65763，#66481，#67105，#67248，#67470，#67638，#68135，#68191，#68211，#68160，#68185，#68127，#68887，#69191, #70961，#68020，#67923，#67963，#68482，#68546，#68593，#68793

• Fixed issues related to Paddle TensorRT conversion and execution. #66932，#66655，#67274，#67504，#65780，#68170，#68647，#68776，#69573，#69598，#69510，#69864，#69885，#70161，#70116，#70791，#70801，#70824，#70939， #71143，#71154，#71163，#71183，#71233，#71287，#71319，#67720，#69671，#70168，#69957

• Fixed issues related to Paddle Inference compilation and linking. #65846, #67081, #63184

• Fixed quantization issues. #67839, #68049, #70099, #64878, #65717, #67552, #67715

• Fixed OneDNN inference issues. #67836, #68021, #68132, #71426, #68057

• Fixed memory issues. #68631, #69129, #70314, #67863

• Paddle Inference supports bug fixes for OpenVINO issues. #70212, #70288,

• Fixed issues related to Pass. #65349，#65421，#65677，#66850，#67443，#67620，#68158，#68642，#68837，#68880，#68935，#69112，#69205，#69242，#69352，#69421，#69690，

• Fixed other issues. #70237, #68173

• Fixed issues related to fused_moe (testing/GEMM/WINT4/multi-architecture compatibility/Bias optional) #67353, #67396, #67717, #67794, #67783

• Fixed issues in the block_attention series (GQA discrepancy/out-of-bounds risk/multi-head support) #67175, #69001, #70763

• Fixed PIR-related issues (layout conversion/BF16 replacement errors) #66977, #67830

• Fixed distributed-related issues (allreduce data type/parameter synchronization) #67449, #69157

• Fixed kernel execution issues (forward-backward conflict/default stream argsort) #67218, #68374

• Other key fixes (reducing the size of the C++ library/fixing RoPE calculation in NeoX format/fixing static graph execution) #66041, #66583, #67580

Other modifications¶

• Code cleanup and maintenance (API deprecation/compilation warning fixes) #68048, #70384

• Third-party integration optimization (OpenVINO submodule management) #70313, #70425

New Features¶

The addition of operations (ops) and improvement of functions on Kunlun Core XPU involve the following ops: flash attention/flash_attn_unpadded, multinomial, matmul, repeat_interleave, logsumexp, index_put_grad, mean_grad, pow, pow_grad, rsqrt, full, rms_norm, rms_norm_grad, put_along_axis, Cumsum, argmin, masked_select/grad, expand_v2/grad, all2all, expand, reduce_sum, reduce_max, reduce_min, moe, fused_linear_param_grad_add, adamw, clip/clip_grad, tan, acos, blha_get_max_len, gather/gather_grad, scatter/scatter_grad, round, index_select/sindex_select_grad, isfinite, isinf, quantize_linear, dequantize_linear, conv3d_transpose, logsumexp_grad, index_add_grad, eye, gather_element, tril, triu, set_value_grad, argmax, take_along_axis, etc #65413, #64846, #65656, #65963, #66143, #66482, #66585, #67077, #67173, #67551, #63989, #67919, #68052, #68176, #68408, #68454, #68478, #68473, #68453, #68770, #68933, #69042, #68713, #69368, #69723, #69767, #69898, #69970, #69771, #70176, #70428, #70573, #70576, #70633, #70114, #70627, #71038, #71132, #71228, #71274, #71364, #71375, #71431, #71451, #67585, #67637, #67914, #67641, #67913, #67955, #68411, #68560, #68423, #68894, #71053, #71047, #69056, #70843, #65653, #68023, #67780, #68622, #67215

Add support for rocsolver and warpctc on Haiguang DCU, and carry out the addition of OPs and improvement of functions. The involved ops include: flash_attention, hipblaslt, fastgelu, multiclass_nms3

#68066, #69457, #68603, #65599, #70587, #71337, #70173

Bug fixes¶

Bug fix for OP on Kunlun Core XPU #65020, #65251, #65418, #65387, #65525, #65613, #65533, #65705, #65915, #66238, #66485, #67349, #67372, #67276, #67460, #67496, #67530, #67828, #68010, #68157, #68172, #68388, #68213, #68501, #68504, #68585, #69229, #69374, #69424, #69440, #69614, #68542, #69990, #70351, #70479, #70431, #70638, #70856, #70974, #70973, #71027, #71062, #71115, #71110, #70858, #71147, #71212, #71361, #71423, #70859, #71492, #71493, #69826, #67341, #68906, #71171

Bug fix for OP on Haiguang DCU #69617, #65716, #66630, #65399

Performance optimization¶

Kunlun Core XPU upgrades the functions of basic components such as streams and optimizes the performance of certain operations. #65102, #69727, #69899, #69942, #70025, #70640

Upgrade of hardware underlying basic libraries¶

The upgrade of the basic library supports Kunlun Core P800, as well as the support for basic components #65494, #65924, #69752, #70835, #65554, #66998, #65278, #70614, #71012, #71178, #71168, #68740, #71100, #65221, #67983

Others¶

Modifications to related modules such as op test #65654, #66233, #66728, #67959, #68169, #68418, #68434, #68445, #68877, #68993, #69006, #70471, #70706, #67777, #65698, #68433, #65689

Bug Fixes¶

• Improve the CI/CD process, fix test cases, resolve compilation and installation issues in different environments, and enhance the stability and cross-environment compatibility of the framework. #65627, #65736, #65900, #66069, #67000, #67312, #67432, #67540, #67670, #68449, #70806, #65665, #65652, #70644, #68119, #68466, #68858, #68788, #68934, #69883, #69924, #71187, #70798, #71248, #70512, #71363, #71438, #71291

Improvement and Upgrade¶

• Environmental upgrade #69491, #66560, #65686, #71177, #71284, #69791, #69349, #70944, #65411

• Pipeline merging #66815, #67306

• Improvement of DCU/NPU/KUNLUN pipeline #67516, #67629, #67987, #69903, #68448, #70401, #71192, #71197, #68027

• Support for Windows environment #70390, #70785, #71286, #71414, #68901

• Improvement of third-party libraries #71419

• Other optimizations are aimed at enhancing CI stability and execution efficiency #67574, #69058, #70610, #67093, #69037, #65213, #65913, #65947, #66479, #71054, #71396

New Features¶

• Added Github Action mechanism #70571, #70626, #71325, #71344, #71353, #71322, #70415, #70465, #70524, #70550, #70564, #70579, #70580, #70963, #71200, #71261, #71265

Discarded¶

• Cleanup of obsolete code and dependencies, including removing Python libraries that are no longer needed and simplifying compilation configurations to reduce maintenance costs #65635, #67542, #67609, #69572, #68150, #67604, #68561, #68904, #67219

Developer-related content¶

• Remove useless debugging code and migrate code #65256, #65782, #65836, #65840, #65862, #65863, #65987, #66547, #66556, #66645, #66646, #66648, #66672, #66783, #66083, #65562, #66564, #66370, #66912, #66913, #66914, #66915, #66664, #66671, #66121, #65907, #65949, #65950, #65954, #66545, #66649, #66900, #66901, #66902, #66903, #66904, #66906, #66907, #66908, #66909, #66549, #66555, #66647, #66898, #66886, #66042, #66043, #66045, #66046, #65826, #65825, #65827, #65829, #65830, #65831, #66081, #66082, #66087, #65980, #65981, #65983, #65985, #65979, #65986, #65988, #65989, #66682, #66717, #65802, #66159, #66147, #66149, #66150, #65798, #65731, #66145, #66086, #65781, #65837, #65828, #65864, #65959, #65706, #66918, #66191, #66689, #66808, #65424, #65452, #65463, #65478, #65339

• Standardize code namespaces #64755, #64765, #64767, #64770, #64775, #64776, #64757, #64780, #64777, #64779, #64758, #64759, #64762

• Modify operator list #66573, #65598, #65100, #65385, #65192, #65118, #65108, #65153, #65465, #65128, #65420, #65099, #65207, #66066, #65400, #65160, #65195, #65445, #65479, #65193, #65401, #66724, #65164, #65466, #65661, #65897, #66022, #65313, #65616, #65588, #65174, #65402, #65154, #65151, #65098, #64953, #65122, #65590, #65152

• The old executor function of the Paddle framework is being phased out #65077, #65340

• Error message prompt optimization #66668, #66675, #66605, #66613, #66507, #66700, #66739, #66719, #66733, #66552, #66548, #66623, #66702, #66705, #66718, #66727, #66860, #66869, #66933, #66939, #66553, #66774, #66794, #66551, #66540, #66617, #66841, #66788, #66954, #66698, #66782, #66844, #66443, #66455, #66517, #66804, #66802, #66536, #66707, #66525, #66753, #66550, #66857, #66471, #66628, #66469, #66775, #66506, #66780, #66953, #66695, #66603, #66491, #66715, #66632, #66594, #66615, #66578, #66534, #66569, #66529, #66530, #66522, #66789, #66600, #66511, #66512, #66527, #66518, #66958, #66532, #65258, #66487, #66876, #66832, #66872, #66830, #66708, #66502, #66521, #66592

Discarded¶

• Clean up abandoned code and useless unit tests #65894, #66165, #66293, #66102, #66442, #66922, #66531, #65518, #66800, #66372, #65902, #65462, #65327, #65189, #65181, #66535, #65383, #65173, #66429, #66386, #66447, #66367, #66160, #65408, #65433, #65481, #65444, #65389, #65663, #65649, #65629, #66142, #65796, #66163, #66291, #65480, #65495, #65498, #65503, #65502, #65501, #65512, #65528, #65472, #65390, #65344, #65384, #65388, #65198, #65248, #65443, #65430