The process of finding the flaws in a system and breaking the system is called "Security Research".
Vulnerabilities are typically found by security researchers. The people who like to find flaws/vulnerabilities in systems are called "Security Researchers".
Like penetration testing, security research can be used for good and evil. Some countries don’t make the distinction and outlaw security research completely.