AWS Glue adds additional 13 new transforms including flag duplicates
AWS Glue now offers 13 new built-in transforms: Flag duplicates in column, Format Phone Number, Format case, Fill with mode, Flag duplicate rows, Remove duplicates, Month name, Is even, Cryptographic Hash, Decrypt, Encrypt, Int to IP and IP to int. AWS Glue is a serverless data integration service that makes it easy for analytics users to discover, prepare, move, and integrate data from multiple sources. With these new transform, ETL developers can quickly build more sophisticated data pipelines without having to write custom code for these common transform tasks.
Each of these new transforms address a unique data processing need. For example, use Remove duplicates, Flag duplicates in column or Flag duplicate rows to highlight or remove the duplicates rows within your dataset, use Cryptographic Hash to apply an algorithm to hash values in the column, encrypt values in the source columns with the Encrypt transform, or decrypt these columns with the Decrypt transform. The new transformations are available for code-based jobs.
These new transforms are now available in all AWS regions where AWS Glue 4.0 is available. Access the AWS Regional Services List for the most up-to-date availability information
To learn more, visit the AWS Glue documentation.