You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 02:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Mar 14, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48916221820,5sleep30-21swapper/323:01:453
4903742760,1chrt490378-21switchtime23:01:540
255279995850,6cyclictest280-21systemd-journal00:00:001
255276995140,8cyclictest509876-21ntpq23:21:450
255276995040,7cyclictest366608-21latency_hist21:00:000
255276995040,7cyclictest366608-21latency_hist21:00:000
255281994838,7cyclictest580632-21gpgv00:35:002
255279994642,2cyclictest322356-21kworker/u8:0+events_unbound22:21:441
255281994534,8cyclictest280-21systemd-journal22:38:142
255276994538,5cyclictest297412-21kworker/u8:2+flush-8:021:21:390
255281994332,9cyclictest390531-21ntpq21:21:472
255281994030,8cyclictest534818-21ntpq23:46:482
255276993933,4cyclictest297412-21kworker/u8:2+flush-8:020:51:510
255276993932,6cyclictest495564-21kworker/u8:0+flush-8:023:41:530
25527699390,37cyclictest580583-21munin-run00:35:000
255279993829,7cyclictest280-21systemd-journal22:57:451
2611042370,3sleep30-21swapper/319:15:023
255279993730,5cyclictest297412-21kworker/u8:2+flush-8:020:06:381
255279993729,6cyclictest321899-21latency_hist20:15:011
255276993730,5cyclictest475608-21kworker/u8:3+flush-8:023:26:510
255279993628,6cyclictest486305-21/usr/sbin/munin23:01:471
2541482363,8sleep10-21swapper/119:06:461
255279993525,8cyclictest569612-21ntpq00:21:461
255276993528,5cyclictest280-21systemd-journal20:17:450
255276993528,5cyclictest240459-21kworker/u8:2+flush-8:019:15:050
255276993525,7cyclictest280-21systemd-journal19:21:440
255279993427,5cyclictest280-21systemd-journal19:46:441
255279993330,2cyclictest476747-21irqcore22:51:451
255279993329,3cyclictest475608-21kworker/u8:3+events_unbound23:21:421
255276993328,4cyclictest282031-21kworker/u8:0+flush-8:019:41:400
255276993326,5cyclictest549801-21ntpq00:01:480
255276993323,8cyclictest544693-21ntpq23:56:470
255279993229,2cyclictest260925-21kthreadcore19:15:031
255279993224,6cyclictest280-21systemd-journal21:31:461
255276993227,3cyclictest312509-21aten2_r5power_v20:06:360
255276993223,6cyclictest465957-21latency_hist22:40:010
255276993212,17cyclictest321939-21kworker/0:1+events21:01:480
255279993127,3cyclictest432226-21irqrtprio22:06:421
255279993127,2cyclictest575361-21latency_hist00:30:011
255276993124,5cyclictest321588-21grep20:11:550
255276993117,12cyclictest386410-21sendmail21:20:000
2550362314,9sleep20-21swapper/219:09:002
50615923024,3sleep2506267-21aten2_r5power_v23:21:352
255279993026,2cyclictest471994-21irqrtprio22:46:411
255279993021,6cyclictest302148-21latency_hist19:55:011
255276993025,3cyclictest397079-21cpuspeed_turbos21:31:380
255276993023,5cyclictest505469-21unixbench_singl23:16:540
255276993016,10cyclictest445968-21apt-get22:20:010
255281992925,2cyclictest267272-21cron19:20:002
255279992923,4cyclictest260248-21base6419:15:001
255279992920,6cyclictest371555-21latency_hist21:05:011
255276992924,3cyclictest500834-21systemd-run23:15:000
255276992923,4cyclictest431157-21kworker/u8:1+events_unbound22:15:000
255276992921,5cyclictest425809-21grep21:56:550
255281992824,2cyclictest521250-21fschecks_count23:36:382
255281992823,4cyclictest401-21dbus-daemon22:20:012
255281992822,4cyclictest286768-21unixbench_singl19:36:542
255279992824,2cyclictest526179-21cut23:41:391
255279992824,2cyclictest391053-21unixbench_singl21:21:541
255279992824,2cyclictest337609-21fschecks_count20:31:391
255279992822,4cyclictest580256-21unixbench_singl00:31:541
255279992822,4cyclictest416001-21unixbench_singl21:46:541
255276992824,3cyclictest332041-21idleruntime-cro20:25:000
255276992824,3cyclictest303437-21kthreadcore19:56:450
255276992823,4cyclictest297412-21kworker/u8:2+flush-8:020:41:390
255276992823,3cyclictest478385-21kthreadcore22:51:470
255276992821,6cyclictest380577-21sshd21:11:440
255276992821,4cyclictest322356-21kworker/u8:0+events_unbound21:26:540
255276992819,6cyclictest280-21systemd-journal23:31:440
25528199278,2cyclictest421rcuog/220:15:012
255281992723,2cyclictest307525-21aten2_r5power_e20:01:392
255281992721,4cyclictest450799-21unixbench_multi22:21:562
255279992723,3cyclictest427137-21irqcore22:01:411
255279992721,4cyclictest361350-21unixbench_multi20:51:541
255276992723,2cyclictest558726-21kthreadcore00:11:440
255276992722,3cyclictest555577-21systemd-run00:10:000
255276992716,3cyclictest0-21swapper/019:50:010
255276992713,11cyclictest575450-21awk00:30:000
255281992622,3cyclictest240459-21kworker/u8:2+flush-8:019:11:152
255281992622,2cyclictest382038-21cpuspeed_turbos21:16:382
255281992616,7cyclictest297385-21cp19:50:012
255279992620,4cyclictest520615-21kworker/u8:1+events_unbound23:55:011
255279992620,4cyclictest272577-21apt-get19:26:331
255276992622,2cyclictest467263-21kthreadcore22:41:430
25527699262,16cyclictest583-21snmpd23:36:350
255276992621,3cyclictest260590-21latency_hist19:15:010
255276992620,4cyclictest495536-21unixbench_singl23:06:530
255276992620,4cyclictest425921-21kworker/u8:3+flush-8:022:26:480
255276992620,4cyclictest339509-21kthreadcore20:31:450
255276992616,7cyclictest280-21systemd-journal23:45:030
255281992522,2cyclictest421601-21aten2_r5power_c21:56:362
255281992521,2cyclictest485637-21unixbench_singl22:56:552
255281992521,2cyclictest282049-21cron19:35:002
255281992521,2cyclictest0-21swapper/222:01:452
25528199252,10cyclictest3431592cut20:36:422
255281992520,3cyclictest322718-21cpuspeed_turbos20:16:382
255279992522,2cyclictest403956-21ps21:36:431
255279992521,2cyclictest0-21swapper/120:21:451
255279992521,2cyclictest0-21swapper/119:41:421
255279992520,3cyclictest377802-21kthreadcore21:11:401
255279992520,3cyclictest0-21swapper/122:30:011
255276992521,2cyclictest482125-21irqrtprio22:56:420
255276992519,4cyclictest536180-21cpuspeed23:51:370
255276992519,4cyclictest250250-21kworker/u8:3+events_unbound21:36:450
4082422240,3sleep375-21kworker/3:1-events21:41:443
255281992421,2cyclictest431856-21fschecks_time22:06:402
255281992420,2cyclictest546119-21aten2_r5power_v00:01:372
255279992420,2cyclictest0-21swapper/123:36:501
255279992420,2cyclictest0-21swapper/122:31:401
255279992420,2cyclictest0-21swapper/100:11:391
255279992419,4cyclictest250250-21kworker/u8:3+flush-8:021:16:401
255279992418,4cyclictest311691-21unixbench_multi20:01:571
255276992420,2cyclictest0-21swapper/021:46:540
255276992419,3cyclictest0-21swapper/022:06:440
255276992418,4cyclictest475524-21unixbench_multi22:46:540
255276992418,4cyclictest250250-21kworker/u8:3+flush-8:021:41:450
255276992416,5cyclictest311681-21grep20:01:580
2550192243,3sleep00-21swapper/019:08:460
255281992319,2cyclictest550995-21aten2_r5power_c00:06:372
255281992318,3cyclictest360781-21grep20:51:462
255281992315,6cyclictest575617-21gpgv00:30:002
255279992319,2cyclictest0-21swapper/123:26:471
255279992319,2cyclictest0-21swapper/100:16:541
255279992318,4cyclictest1-21systemd22:45:001
255279992316,5cyclictest439500-21cut22:11:441
255279992316,3cyclictest0-21swapper/121:06:361
255279992315,6cyclictest555550-21latency_hist00:10:011
25527699237,3cyclictest2785352tr19:31:430
25527699232,20cyclictest0-21swapper/000:36:520
25527699232,20cyclictest0-21swapper/000:36:520
255276992319,2cyclictest0-21swapper/020:36:530
255276992318,4cyclictest322356-21kworker/u8:0+flush-8:022:22:060
255276992317,4cyclictest417972-21kthreadcore21:51:440
255276992314,6cyclictest430359-21ntpq22:01:490
255276992314,5cyclictest0-21swapper/000:16:360
255276992314,4cyclictest0-21swapper/019:26:460
25528199222,14cyclictest287451-21apt-get19:41:372
255281992219,2cyclictest561144-21fschecks_count00:16:392
255281992219,2cyclictest486414-21aten2_r5power_p23:01:372
255281992218,3cyclictest541178-21cpuspeed_turbos23:56:382
255281992218,2cyclictest372188-21fschecks_count21:06:392
255279992219,2cyclictest1-21systemd23:50:011
255279992218,2cyclictest0-21swapper/123:15:001
255279992218,2cyclictest0-21swapper/119:36:411
255279992217,3cyclictest336633-21unixbench_multi20:26:531
255279992217,3cyclictest0-21swapper/119:56:451
255279992216,4cyclictest495512-21unixbench_multi23:06:531
255279992215,5cyclictest409-21smartd22:19:471
255276992213,6cyclictest280-21systemd-journal20:26:440
25528199212,14cyclictest406769-21apt-get21:41:372
25528199212,13cyclictest411680-21apt-get21:46:362
255281992118,2cyclictest511440-21fschecks_time23:26:382
255281992117,2cyclictest377808-21kthreadcore21:11:402
255281992116,4cyclictest297413-21kworker/u8:4+flush-8:019:51:522
255281992112,7cyclictest517518-21kthreadcore23:31:442
255279992117,3cyclictest880-21mta-sts-daemon21:56:411
255279992116,4cyclictest550263-21unixbench_multi00:01:541
255279992116,3cyclictest0-21swapper/121:26:551
255279992116,3cyclictest0-21swapper/100:40:001
255279992116,3cyclictest0-21swapper/100:40:001
255279992115,4cyclictest272215-21latency_hist19:25:011
255279992115,4cyclictest0-21swapper/121:51:521
255279992115,4cyclictest0-21swapper/120:56:401
255279992115,4cyclictest0-21swapper/120:56:401
255276992115,4cyclictest520615-21kworker/u8:1+flush-8:000:21:460
255276992114,5cyclictest286387-21ps19:36:510
25527699211,11cyclictest296956-21apt-get19:50:030
255276992110,9cyclictest355947-21ntpq20:46:490
25528199202,13cyclictest496199-21apt-get23:11:372
25528199202,13cyclictest436439-21apt-get22:11:362
255281992016,2cyclictest367340-21fschecks_time21:01:402
255281992015,3cyclictest0-21swapper/219:56:462
255281992013,5cyclictest505040-21ntpq23:16:502
255279992017,2cyclictest410434-21needreboot21:41:471
255279992016,2cyclictest0-21swapper/122:36:431
255279992014,4cyclictest0-21swapper/119:31:411
255276992017,2cyclictest255270-21cyclictest21:06:410
255276992015,3cyclictest456620-21cpuspeed22:31:380
2509702203,9sleep30-21swapper/319:06:413
25528199192,12cyclictest581126-21apt-get00:36:362
25528199192,12cyclictest581126-21apt-get00:36:362
25528199192,12cyclictest451496-21apt-get22:26:362
255281991916,2cyclictest495825-21ls23:10:002
255281991916,2cyclictest332723-21fschecks_time20:26:382
255281991915,2cyclictest456784-21fschecks_time22:31:402
255281991915,2cyclictest0-21swapper/222:41:372
255281991912,5cyclictest556673-21irqrtprio00:11:422
25528199190,17cyclictest268365-21sed19:21:422
255279991915,3cyclictest0-21swapper/120:41:371
255279991915,2cyclictest0-21swapper/123:16:361
255279991913,4cyclictest0-21swapper/123:31:401
255279991913,4cyclictest0-21swapper/120:46:351
3839332180,2chrt75-21kworker/3:1+events21:16:443
25528199182,12cyclictest416660-21apt-get21:51:372
255281991815,2cyclictest476562-21fschecks_time22:51:422
255281991814,2cyclictest0-21swapper/222:46:482
255281991814,2cyclictest0-21swapper/200:25:002
255281991813,4cyclictest540574-21cron23:55:002
255281991811,5cyclictest394391-21kthreadcore21:26:452
255279991813,4cyclictest0-21swapper/120:36:471
25528199172,11cyclictest405575-21ntpq21:36:472
255281991714,2cyclictest530758-21latency_hist23:45:012
255281991713,2cyclictest0-21swapper/220:41:532
255281991713,2cyclictest0-21swapper/220:21:372
255281991711,4cyclictest341630-21unixbench_singl20:31:542
4465902162,9sleep30-21swapper/322:21:373
2785302160,2sleep3481ktimers/319:31:433
25528199167,7cyclictest352797-21sed20:46:432
25528199162,2cyclictest421rcuog/219:30:002
255281991613,1cyclictest0-21swapper/220:58:372
255281991613,1cyclictest0-21swapper/220:58:372
255281991612,2cyclictest0-21swapper/220:06:482
255281991610,4cyclictest0-21swapper/221:35:012
25527999166,2cyclictest0-21swapper/120:17:541
4371082152,9sleep30-21swapper/322:11:423
3065232155,6sleep30-21swapper/319:56:563
4276422142,8sleep30-21swapper/322:01:433
2771312142,8sleep30-21swapper/319:30:003
5646352132,7sleep30-21swapper/300:16:493
5239662132,7sleep30-21swapper/323:36:453
5755002122,5sleep30-21swapper/300:30:023
5735852122,6sleep30-21swapper/300:26:483
5658382122,6sleep30-21swapper/300:21:353
5506232122,5sleep30-21swapper/300:05:013
5416782122,6sleep30-21swapper/323:56:433
5360202120,8sleep30-21swapper/323:51:373
5347962121,7sleep30-21swapper/323:46:483
5260302123,5sleep30-21swapper/323:41:383
5159782122,6sleep30-21swapper/323:31:353
4916582122,6sleep30-21swapper/323:06:393
4764712122,6sleep30-21swapper/322:51:433
4620192123,5sleep30-21swapper/322:36:423
4451382122,6sleep30-21swapper/322:16:483
4212212122,5sleep30-21swapper/321:55:013
3965022122,5sleep30-21swapper/321:30:013
3956382123,6sleep30-21swapper/321:26:483
3865942122,6sleep30-21swapper/321:20:033
3799662122,6sleep30-21swapper/321:11:443
3757572122,6sleep30-21swapper/321:06:513
3567712122,5sleep30-21swapper/320:50:013
3470492122,5sleep30-21swapper/320:40:033
3382272122,6sleep30-21swapper/320:31:433
3320282122,5sleep30-21swapper/320:25:013
3277822122,6sleep30-21swapper/320:21:403
3221812122,5sleep30-21swapper/320:15:023
3116722122,6sleep30-21swapper/320:01:583
2916852122,6sleep30-21swapper/319:41:563
2821952122,6sleep30-21swapper/319:35:013
2558002122,6sleep30-21swapper/319:11:383
5814112112,5sleep30-21swapper/300:36:393
5814112112,5sleep30-21swapper/300:36:393
5597942112,5sleep30-21swapper/300:11:523
5460102112,5sleep30-21swapper/300:01:363
5110622112,5sleep30-21swapper/323:26:333
5066082112,5sleep30-21swapper/323:21:403
5010952112,6sleep30-21swapper/323:15:503
4960562113,5sleep30-21swapper/323:10:053
4812612112,6sleep30-21swapper/322:55:093
4712792112,5sleep30-21swapper/322:46:373
4665892112,5sleep30-21swapper/322:41:413
4573602112,5sleep30-21swapper/322:31:443
4513532112,5sleep30-21swapper/322:25:023
4315102112,5sleep30-21swapper/322:06:373
4166242112,5sleep30-21swapper/321:51:283
4115372112,5sleep30-21swapper/321:45:023
4056682112,5sleep30-21swapper/321:36:513
3668762112,5sleep30-21swapper/321:00:283
3665862112,5sleep30-21swapper/321:00:003
3665862112,5sleep30-21swapper/321:00:003
3521662112,5sleep30-21swapper/320:46:363
3446642112,5sleep30-21swapper/320:36:453
3172402112,5sleep30-21swapper/320:10:453
3162512112,5sleep30-21swapper/320:06:513
2971552112,5sleep30-21swapper/319:50:023
2927702112,5sleep30-21swapper/319:46:413
2676732112,5sleep30-21swapper/319:21:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional