You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 11:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Apr 13, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3440367996046,9cyclictest420-21avahi-daemon23:40:262
3440367995631,21cyclictest3441838-21cron19:15:002
3440367995631,21cyclictest3441838-21cron19:15:002
3440367995243,6cyclictest3500912-21systemd-run21:40:022
3440365995124,7cyclictest12450irq/33-eth019:25:340
3440367995030,16cyclictest3599188-21munin-run23:40:012
344036599495,35cyclictest3518545-21ssh22:00:390
344036599494,38cyclictest3538963-21ssh22:25:370
3440366994839,6cyclictest3533666-21munin-run22:20:011
3440367994741,4cyclictest0-21swapper/222:00:322
344036799466,35cyclictest3625055-21ssh00:10:392
344036799464,34cyclictest3514309-21ssh21:55:342
3440367994636,8cyclictest3554159-21sh22:45:012
3440367994611,32cyclictest3635856-21sh00:24:402
344036699468,30cyclictest3534950-21ssh22:20:401
3440366994638,5cyclictest421-21dbus-daemon23:05:011
344036799454,21cyclictest3645381-21ssh00:35:352
344036799451,14cyclictest3559400-21ntpq22:50:372
3440366994536,6cyclictest1-21systemd00:05:011
344036699451,7cyclictest171rcu_preempt19:25:351
344036799441,16cyclictest3448821-21ntpq19:35:372
344036699444,36cyclictest3506180-21ssh21:45:371
3440366994438,3cyclictest0-21swapper/121:35:321
3440366994435,6cyclictest3640672-21aten_r2power_en00:30:261
344036599443,13cyclictest3510378-21ntpq21:50:420
344036599443,13cyclictest3510378-21ntpq21:50:420
344036799438,32cyclictest3549329-21ssh22:38:412
344036799435,30cyclictest3485745-21ssh21:20:372
344036799434,7cyclictest3536892-21sh22:23:232
3440367994334,6cyclictest0-21swapper/221:10:022
344036799432,6cyclictest3506290-21ntpq21:45:422
344036799432,14cyclictest3456327-21ntpq20:00:382
344036799431,11cyclictest3503583-21ssh21:42:232
344036799431,11cyclictest3493848-21ntpq21:30:362
344036799426,33cyclictest3528599-21sh22:13:132
344036799423,7cyclictest3453366-21ntpq19:50:412
344036799422,6cyclictest3471305-21ntpq20:50:392
344036799422,6cyclictest3442723-21ntpq19:15:352
344036799422,6cyclictest3442723-21ntpq19:15:352
344036799422,15cyclictest3629093-21ntpq00:15:372
344036799421,11cyclictest3484156-21ssh21:19:182
344036799421,10cyclictest3525101-21ssh22:09:212
344036799420,11cyclictest3575744-21ntpq23:10:332
3440366994233,6cyclictest3517393-21latency_hist22:00:021
3440365994234,5cyclictest1499373-21cupsd23:38:140
3440365994229,10cyclictest420-21avahi-daemon21:55:500
344036799418,3cyclictest3556928-21ssh22:47:332
344036799417,4cyclictest3585705-21ssh23:22:432
344036799412,6cyclictest3530695-21ntpq22:15:342
344036799412,6cyclictest3465251-21ntpq20:30:362
344036799411,9cyclictest3639347-21ssh00:28:312
344036799411,9cyclictest3591263-21ssh23:30:042
344036799411,5cyclictest3444210-21ntpq19:20:322
344036799411,15cyclictest3469911-21ps20:45:412
344036799411,11cyclictest3463856-21ntpq20:25:382
344036799411,11cyclictest3460773-21ntpq20:15:352
344036799411,11cyclictest3460773-21ntpq20:15:352
344036799411,10cyclictest3613964-21ssh23:57:042
344036799411,10cyclictest3582277-21ssh23:19:022
344036799410,10cyclictest3619280-21ssh00:04:172
3440366994133,5cyclictest0-21swapper/122:10:361
3440366994132,6cyclictest3562354-21ssh22:55:011
3440366994132,5cyclictest0-21swapper/122:05:361
3440366994132,4cyclictest0-21swapper/123:30:391
3440366994131,6cyclictest3584193-21proc_pri23:20:421
344036699412,33cyclictest3600491-21ssh23:40:401
3440365994135,4cyclictest3648338-21munin-run00:40:010
3440365994132,6cyclictest151ktimers/023:05:440
3440365994130,4cyclictest1051-21mta-sts-daemon22:25:000
344036799408,30cyclictest3490994-21ssh21:26:472
344036799401,9cyclictest3568625-21ssh23:01:252
344036799401,9cyclictest3545580-21ssh22:34:222
344036799401,9cyclictest3545580-21ssh22:34:222
344036799401,9cyclictest3539995-21ssh22:26:312
344036799401,8cyclictest3590152-21sh23:28:242
344036799401,8cyclictest3570847-21sh23:05:112
344036799401,20cyclictest3510391-21ssh21:50:422
344036799401,20cyclictest3510391-21ssh21:50:422
344036799401,11cyclictest3457754-21ntpq20:05:332
344036799401,11cyclictest3457754-21ntpq20:05:332
344036799401,10cyclictest3565691-21ssh22:58:402
344036799400,9cyclictest3478811-21ssh21:11:532
3440366994032,5cyclictest3481561-21needreboot21:15:341
3440366994032,5cyclictest3476388-21diskmemload22:45:021
344036699402,32cyclictest3575900-21ssh23:10:391
344036599405,21cyclictest3502061-21ssh21:40:350
3440365994033,5cyclictest3616049-21aten_r2power_cu00:00:250
3440365994033,4cyclictest0-21swapper/000:00:010
3440365994023,11cyclictest12350irq/32-eth023:20:330
3440365994018,12cyclictest12350irq/32-eth020:00:380
344036799394,12cyclictest3451755-21ntpq19:45:362
344036799392,11cyclictest3474275-21ntpq21:00:352
344036799391,9cyclictest3621963-21ssh00:06:382
344036799391,6cyclictest3450270-21ntpq19:40:322
344036799391,11cyclictest3466755-21ntpq20:35:352
344036799391,10cyclictest3610067-21ssh23:52:262
344036799391,10cyclictest3447357-21ntpq19:30:372
344036699399,7cyclictest3489799-21ssh21:25:361
3440366993933,3cyclictest3453868-21kworker/u8:2-xprtiod20:40:011
3440366993932,4cyclictest1051-21mta-sts-daemon23:15:491
3440366993929,5cyclictest0-21swapper/121:20:421
344036599394,31cyclictest3608552-21ssh23:50:370
3440365993931,5cyclictest420-21avahi-daemon21:21:540
3440365993930,6cyclictest1499375-21cups-browsed23:40:150
3440365993929,7cyclictest3533392-21sshd22:19:320
344036599392,32cyclictest151ktimers/019:20:380
344036799389,26cyclictest3468802-21/usr/sbin/munin20:40:462
344036799388,8cyclictest421-21dbus-daemon21:00:012
344036799381,9cyclictest3603547-21ssh23:45:052
3440366993831,4cyclictest3608128-21fschecks_time23:50:301
3440366993831,3cyclictest0-21swapper/122:35:361
3440366993830,5cyclictest3459877-21cron20:15:001
3440366993828,7cyclictest420-21avahi-daemon22:48:311
3440366993828,6cyclictest420-21avahi-daemon00:07:101
344036799374,9cyclictest3445739-21apt-get19:25:332
344036799373,7cyclictest3462296-21ntpq20:20:362
3440367993719,14cyclictest0-21swapper/220:10:462
344036799370,8cyclictest3642572-21ssh00:31:522
3440366993730,3cyclictest0-21swapper/123:35:361
3440366993729,5cyclictest3518172-21tr22:00:311
3440366993728,6cyclictest3444856-21latency_hist19:25:001
3440366993727,5cyclictest0-21swapper/123:45:361
344036599374,26cyclictest3526682-21ssh22:10:370
344036599374,25cyclictest3579976-21ssh23:15:410
3440365993728,7cyclictest3475699-21needreboot21:05:330
344036799361,11cyclictest3454804-21ntpq19:55:362
3440366993629,3cyclictest3550065-21kworker/u8:7-rpciod23:10:021
3440366993628,5cyclictest3636035-21cron00:25:011
3440366993628,5cyclictest3562950-21aten_r2power_vo22:55:261
3440366993628,5cyclictest3542881-21missed_timers22:30:321
3440366993628,5cyclictest3542881-21missed_timers22:30:321
3440366993625,5cyclictest0-21swapper/121:10:361
344036699362,21cyclictest3476388-21diskmemload23:25:331
3440365993629,3cyclictest0-21swapper/000:10:000
3440366993528,3cyclictest0-21swapper/100:40:011
3440366993525,7cyclictest261-21systemd-journal00:16:321
3440366993525,5cyclictest0-21swapper/121:30:421
344036599353,27cyclictest3604532-21ssh23:45:380
3440365993530,3cyclictest1056-21mta-sts-daemon19:40:380
3440365993526,6cyclictest151ktimers/020:20:400
3440365993525,7cyclictest3505999-21missed_timers21:45:320
3440365993525,7cyclictest0-21swapper/021:30:370
3440366993429,2cyclictest0-21swapper/119:40:011
344036599348,20cyclictest3563657-21sh22:55:410
3440365993428,4cyclictest3453912-21systemd-run19:55:010
3440365993427,4cyclictest1054-21mta-sts-daemon22:37:220
3440365993426,6cyclictest3484525-21cron21:20:010
3440365993425,7cyclictest3473361-21cron21:00:000
344036599342,24cyclictest3633167-21ssh00:20:380
3440366993329,2cyclictest0-21swapper/121:00:011
3440366993325,5cyclictest3510075-21missed_timers21:50:331
3440366993325,5cyclictest3510075-21missed_timers21:50:331
3440366993325,5cyclictest1055-21mta-sts-daemon21:40:371
3440366993324,6cyclictest3451420-21fschecks_count19:45:301
344036599335,24cyclictest3551375-21sh22:40:420
3440365993327,4cyclictest1054-21mta-sts-daemon21:25:360
3440365993327,4cyclictest0-21swapper/000:15:330
3440365993326,5cyclictest3470491-21latency_hist20:50:020
3440365993326,4cyclictest3475080-21latency_hist21:05:020
3440365993326,4cyclictest151ktimers/023:25:390
3440365993325,6cyclictest3544595-21sshd22:32:300
3440365993325,6cyclictest3544595-21sshd22:32:300
3440365993325,5cyclictest3644271-21true00:35:010
3440365993325,5cyclictest3554891-21sort22:45:280
3440365993324,6cyclictest3566452-21kworker/u8:4+xprtiod23:10:470
344036599332,23cyclictest3637418-21ssh00:25:430
3440365993320,10cyclictest421-21dbus-daemon19:35:010
3440366993227,2cyclictest0-21swapper/122:30:011
3440366993226,3cyclictest0-21swapper/121:10:021
3440366993224,5cyclictest3456898-21idleruntime-cro20:05:011
3440366993223,6cyclictest3624839-21needreboot00:10:321
3440366993221,7cyclictest548-21gdbus19:15:011
3440366993221,7cyclictest548-21gdbus19:15:011
3440365993225,3cyclictest0-21swapper/021:35:060
344036599322,25cyclictest3559487-21ssh22:50:400
3440365993216,11cyclictest12350irq/32-eth022:05:360
3440366993125,3cyclictest0-21swapper/120:30:011
3440366993125,3cyclictest0-21swapper/120:25:001
3440366993125,3cyclictest0-21swapper/100:25:371
3440366993124,4cyclictest3455260-21tail19:55:461
3440366993124,3cyclictest0-21swapper/120:45:371
3440366993123,5cyclictest3460282-21aten_r2power_vo20:15:261
3440366993123,5cyclictest3460282-21aten_r2power_vo20:15:261
3440366993122,4cyclictest0-21swapper/120:40:331
3440365993125,3cyclictest3591032-21kworker/u8:5-nfsiod23:35:010
3440365993123,6cyclictest3624506-21fschecks_count00:10:280
3440365993117,4cyclictest0-21swapper/020:50:330
344036799301,7cyclictest3440371-21true19:10:012
3440366993018,3cyclictest3476388-21diskmemload23:55:341
3440365993023,3cyclictest0-21swapper/023:00:340
3440365993015,9cyclictest151ktimers/019:45:360
344036699292,16cyclictest421-21dbus-daemon21:00:371
3440366992921,5cyclictest439-21systemd-logind20:35:011
3440365992922,5cyclictest3448406-21fschecks_count19:35:270
3440365992919,7cyclictest3480447-21cron21:15:000
3440365992917,10cyclictest0-21swapper/019:55:280
3440366992822,3cyclictest0-21swapper/120:10:001
3440366992822,3cyclictest0-21swapper/120:10:001
3440365992822,3cyclictest0-21swapper/019:15:390
3440365992822,3cyclictest0-21swapper/019:15:390
3440366992720,5cyclictest0-21swapper/119:40:451
3440365992720,4cyclictest1051-21mta-sts-daemon20:40:330
3440366992620,3cyclictest0-21swapper/119:30:281
3440365992621,3cyclictest151ktimers/020:35:390
3440365992618,5cyclictest151ktimers/020:30:390
3440366992517,5cyclictest3440306-21kworker/u8:2+xprtiod19:20:001
3440366992517,5cyclictest3440306-21kworker/u8:2+xprtiod19:20:001
3440365992517,5cyclictest516-21in:imuxsock20:30:010
3440365992515,7cyclictest3461480-21latency_hist20:20:010
3440365992515,7cyclictest3461480-21latency_hist20:20:010
3440365992514,8cyclictest3441837-21cron19:15:000
3440365992514,8cyclictest3441837-21cron19:15:000
3440366992315,5cyclictest3458380-21kworker/u8:6-xprtiod20:50:301
3440365992319,2cyclictest0-21swapper/020:10:280
3440366992218,2cyclictest0-21swapper/119:50:411
3440365992211,8cyclictest0-21swapper/020:05:250
3440365992211,8cyclictest0-21swapper/020:05:250
344036599211,9cyclictest3440425-21wc19:10:020
3440366991914,2cyclictest0-21swapper/119:10:011
344036899172,8cyclictest0-21swapper/323:15:393
344036899172,8cyclictest0-21swapper/322:30:403
344036899172,8cyclictest0-21swapper/322:30:403
344036899171,8cyclictest0-21swapper/322:10:353
344036899163,7cyclictest0-21swapper/323:55:393
344036899162,7cyclictest0-21swapper/321:55:383
344036899162,6cyclictest0-21swapper/300:25:383
344036899161,9cyclictest0-21swapper/321:30:383
344036899161,9cyclictest0-21swapper/321:20:373
344036899161,9cyclictest0-21swapper/300:15:373
344036899152,7cyclictest0-21swapper/323:35:343
344036899152,7cyclictest0-21swapper/300:30:393
344036899152,7cyclictest0-21swapper/300:05:373
344036899151,8cyclictest0-21swapper/322:15:353
344036899151,7cyclictest0-21swapper/322:50:423
344036899151,6cyclictest0-21swapper/322:55:343
344036899151,6cyclictest0-21swapper/320:00:353
344036899142,8cyclictest0-21swapper/322:20:363
344036899142,7cyclictest0-21swapper/322:40:303
344036899142,7cyclictest0-21swapper/322:35:333
344036899142,6cyclictest0-21swapper/323:40:403
344036899142,6cyclictest0-21swapper/323:20:393
344036899142,6cyclictest0-21swapper/321:50:313
344036899142,6cyclictest0-21swapper/321:50:313
344036899142,6cyclictest0-21swapper/320:45:353
344036899141,8cyclictest0-21swapper/320:10:413
344036899141,8cyclictest0-21swapper/319:25:333
344036899141,8cyclictest0-21swapper/319:20:363
344036899141,8cyclictest0-21swapper/300:35:353
344036899141,7cyclictest0-21swapper/322:45:343
344036899141,7cyclictest0-21swapper/321:40:343
344036899141,7cyclictest0-21swapper/321:35:373
344036899141,7cyclictest0-21swapper/319:40:383
344036899141,7cyclictest0-21swapper/300:20:363
344036899141,6cyclictest0-21swapper/320:30:363
344036899140,7cyclictest0-21swapper/321:15:383
344036899134,7cyclictest0-21swapper/323:45:373
344036899134,7cyclictest0-21swapper/322:00:383
344036899133,5cyclictest0-21swapper/323:10:343
344036899132,7cyclictest0-21swapper/322:05:373
344036899132,6cyclictest0-21swapper/319:45:383
344036899131,7cyclictest0-21swapper/323:25:353
344036899131,7cyclictest0-21swapper/322:25:363
344036899131,7cyclictest0-21swapper/321:45:423
344036899131,7cyclictest0-21swapper/320:35:383
344036899131,7cyclictest0-21swapper/300:00:363
344036899131,6cyclictest0-21swapper/323:50:373
344036899131,6cyclictest0-21swapper/323:05:363
344036899131,6cyclictest0-21swapper/319:55:353
344036899123,5cyclictest0-21swapper/321:10:363
344036899122,5cyclictest0-21swapper/321:00:413
344036899122,5cyclictest0-21swapper/319:10:373
344036899122,5cyclictest0-21swapper/319:10:373
344036899121,7cyclictest0-21swapper/320:25:393
344036899121,6cyclictest0-21swapper/321:05:373
344036899121,6cyclictest0-21swapper/320:50:413
344036899121,6cyclictest0-21swapper/320:15:363
344036899121,6cyclictest0-21swapper/320:15:363
344036899112,5cyclictest0-21swapper/321:25:293
344036899112,5cyclictest0-21swapper/300:10:293
344036899111,6cyclictest0-21swapper/323:30:383
344036899111,6cyclictest0-21swapper/323:00:423
344036899111,6cyclictest0-21swapper/319:35:363
344036899111,5cyclictest0-21swapper/320:55:343
344036899111,5cyclictest0-21swapper/320:05:333
344036899111,5cyclictest0-21swapper/320:05:333
344036899111,5cyclictest0-21swapper/319:15:333
344036899111,5cyclictest0-21swapper/319:15:333
344036899101,5cyclictest0-21swapper/320:40:333
344036899101,4cyclictest0-21swapper/319:50:293
34403689991,4cyclictest0-21swapper/319:30:283
34403689991,3cyclictest0-21swapper/320:20:273
34403689981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional