The purpose of this study was to assess the effects of acute caffeine supplementation on physical performance and perceived exertion during taekwondo-specific tasks in male and female athletes with varying expertise. In a double-blinded, randomized, placebo-controlled crossover study design, 52 young athletes from elite (n = 32; 16 males and 16 females) and sub-elite competitive level (n = 20; 10 males and 10 females) participated. Athletes performed taekwondo-specific tasks including the taekwondo-specific agility test (TSAT), 10 s frequency speed of kick test (FSKT-10 s) and multi-bout FSKT (FSKT-multi) under the following conditions: (1) Caffeine (CAF; 3 mg kg−1), placebo (PLA), and no supplement control (CON). Session rating of perceived exertion (s-RPE) was determined after the tests. Findings show that regardless of condition, males performed better than females (p < 0.05) and elite athletes had superior performance compared to their sub-elite counterparts (p < 0.05). For the TSAT (p < 0.001), FSKT-10s (p < 0.001), and FSKT-multi (p < 0.001), CAF enhanced performance in elite female athletes compared to sub-elite females. Likewise, CAF ingestion resulted in superior performance in elite males compared to sub-elite males for FSKT-10s (p = 0.003) and FSKT-multi (p < 0.01). The ergogenic potential of CAF during taekwondo-specific tasks appears to be related to a competitive level, with greater benefits in elite than sub-elite athletes.