Help Needed: Redirects for Migrated WordPress to Beehiiv Site

I've migrated my site from WordPress to Beehiiv and updated the DNS to point to Beehiiv. The old site is still accessible at its previous URL that Cloudways hosting gives. However, all my previously published posts are returning 404 errors because Beehiiv adds a /p/ prefix to the URLs. In my old Wordpress site it was just blog.com/blog-post I've attempted to add the necessary rewrite rules to the .htaccess file on the old site, but the redirects aren't functioning as expected. Could someone assist me in ensuring that the .htaccess file is correctly configured and that the server is properly handling these redirects? Additionally, should I remove the old posts to prevent potential duplicate content issues, or is there a better approach to handle this situation?