The discussion also explored how the responsibilities of product managers could change as generative AI systems become part of the development process. Ostrovskiy wrote: “The job becomes less about coordination and more about 1) understanding real user problems, 2) defining what ‘success’ means in an AI system, and 3) building evals and feedback loops so you can tell if a new model configuration is actually better than the last one.” He added that curiosity about how AI systems behave may become a core skill across multiple roles: “The advantage goes to people who are curious about system behavior and who like building, regardless of whether their title says PM, engineer, designer or something else.” The conversation also included advice for students learning how to evaluate AI systems: “Build something with one foundation model, then swap in a different model or prompt configuration and force yourself to decide if it’s better. When you’re a student looking to become a better PM, even a simple spreadsheet of use cases plus a qualitative rubric counts as an eval.”
