You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 06:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Tue Aug 04, 2026 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
49436021980,3sleep3575-21kworker/3:2+events21:23:003
6768962920,2sleep3481ktimers/300:27:573
5762342650,5sleep30-21swapper/322:43:083
6138092590,2sleep2371rcuc/223:22:592
4177772590,5chrt0-21swapper/020:05:010
360415995549,4cyclictest3166996-21kworker/u8:1+events_unbound19:30:001
360415995549,4cyclictest3166996-21kworker/u8:1+events_unbound19:30:001
360415994943,5cyclictest487256-21kworker/u8:2+events_unbound21:28:101
360415994943,4cyclictest539908-21kworker/u8:3+events_unbound22:15:001
360415994943,4cyclictest3166996-21kworker/u8:1+events_unbound19:18:091
360415994940,6cyclictest397831-21latency_hist19:45:021
360412994940,6cyclictest377730-21base6419:25:000
360415994843,3cyclictest482087-21kworker/u8:3+events_unbound21:18:071
360415994841,5cyclictest457136-21kworker/u8:1+flush-8:021:07:531
360415994740,5cyclictest487256-21kworker/u8:2+flush-8:022:28:061
360412994721,23cyclictest487029-21latency_hist21:15:010
360415994640,4cyclictest539908-21kworker/u8:3+events_unbound23:08:051
360412994319,20cyclictest556589-21latency_hist22:25:000
360415994236,4cyclictest422786-21kworker/u8:3+events_unbound20:43:051
360415994235,5cyclictest487256-21kworker/u8:2+flush-8:021:57:551
360412994235,5cyclictest377579-21apt-get19:25:020
360412994235,5cyclictest377579-21apt-get19:25:020
360412994227,13cyclictest616353-21latency_hist23:25:000
360415994135,4cyclictest412664-21kworker/u8:2+flush-8:020:58:051
360415994134,6cyclictest600995-21kworker/u8:0+flush-8:023:22:581
360415994133,6cyclictest432431-21latency_hist20:20:011
360412994133,5cyclictest536783-21latency_hist22:05:010
360412994126,13cyclictest275-21systemd-journal00:30:000
360412994122,17cyclictest676049-21latency_hist00:25:000
360415994035,4cyclictest621639-21kworker/u8:3+flush-8:023:48:051
360415994034,4cyclictest457136-21kworker/u8:1+flush-8:021:13:101
360415993934,4cyclictest539908-21kworker/u8:3+events_unbound22:38:101
360412993924,13cyclictest590605-21ntpq22:58:070
360412993922,13cyclictest0-21swapper/020:13:050
360412993921,16cyclictest438112-21fschecks_count20:27:560
360412993921,16cyclictest438112-21fschecks_count20:27:550
360415993833,4cyclictest457136-21kworker/u8:1+events_unbound20:53:031
360415993833,3cyclictest482087-21kworker/u8:3+events_unbound21:48:091
360415993832,5cyclictest539908-21kworker/u8:3+flush-8:022:58:031
360415993832,4cyclictest482087-21kworker/u8:3+flush-8:021:43:031
360412993822,14cyclictest497881-21fschecks_count21:27:550
360416993728,7cyclictest626319-21apt-get23:35:002
360415993733,3cyclictest571719-21kworker/u8:2+flush-8:023:27:541
360415993731,5cyclictest571719-21kworker/u8:2+flush-8:022:48:081
360415993729,6cyclictest462079-21latency_hist20:50:011
36041299373,25cyclictest0-21swapper/021:02:500
360412993731,4cyclictest401-21dbus-daemon23:15:000
360412993719,16cyclictest482054-21cron21:10:000
360412993715,19cyclictest407694-21latency_hist19:55:010
360415993631,4cyclictest539908-21kworker/u8:3+events_unbound22:33:061
360415993630,4cyclictest539908-21kworker/u8:3+events_unbound22:22:521
360415993630,4cyclictest539908-21kworker/u8:3+events_unbound22:17:521
360415993629,5cyclictest482087-21kworker/u8:3+flush-8:021:38:021
360415993629,5cyclictest422786-21kworker/u8:3+flush-8:020:23:041
360415993625,9cyclictest476897-21apt-get21:05:001
36041299363,26cyclictest0-21swapper/020:32:500
36041299363,26cyclictest0-21swapper/000:17:490
360412993627,7cyclictest432888-21/usr/sbin/munin20:23:020
360412993627,6cyclictest452165-21latency_hist20:40:020
360412993619,15cyclictest511813-21unixbench_singl21:38:090
360415993531,3cyclictest651539-21kworker/u8:4+flush-8:000:18:071
360415993530,4cyclictest317305-21kworker/u8:3+flush-8:019:13:081
360415993530,4cyclictest3166996-21kworker/u8:1+events_unbound19:22:541
360415993530,3cyclictest422786-21kworker/u8:3+flush-8:020:13:091
360415993530,3cyclictest387009-21kworker/u8:0+events_unbound19:57:521
360415993529,5cyclictest539908-21kworker/u8:3+flush-8:022:08:051
360415993528,5cyclictest566427-21kworker/u8:1+flush-8:022:43:011
36041299353,27cyclictest0-21swapper/023:27:500
36041299353,24cyclictest0-21swapper/020:57:510
36041299353,17cyclictest0-21swapper/021:22:510
360412993519,14cyclictest0-21swapper/019:47:550
360412993518,15cyclictest665769-21switchtime00:13:090
360412993518,15cyclictest626133-21unixbench_singl23:33:090
360412993518,15cyclictest0-21swapper/022:40:010
36041699349,12cyclictest393356-21apt-get19:42:512
360415993429,4cyclictest571719-21kworker/u8:2+events_unbound23:53:111
360415993429,3cyclictest413828-21irqrtprio20:02:581
360415993428,5cyclictest507085-21kworker/u8:1+flush-8:022:03:081
360415993426,6cyclictest631966-21/usr/sbin/munin23:43:081
36041299342,24cyclictest571-21snmpd23:42:510
36041299342,22cyclictest0-21swapper/021:16:540
360412993418,14cyclictest422305-21unixbench_multi20:08:110
360412993417,15cyclictest581873-21aten2_r5power_c22:52:550
360412993417,15cyclictest0-21swapper/000:08:090
360412993413,19cyclictest537833-21ps22:07:570
360412993412,18cyclictest387833-21grep19:35:000
360415993327,4cyclictest651538-21kworker/u8:1+events_unbound00:33:071
360415993327,4cyclictest571719-21kworker/u8:2+flush-8:022:52:571
360415993324,7cyclictest606279-21cat23:15:001
36041299334,24cyclictest0-21swapper/021:32:510
36041299333,27cyclictest0-21swapper/019:37:490
36041299333,26cyclictest0-21swapper/022:32:510
36041299333,25cyclictest0-21swapper/020:17:490
360412993328,3cyclictest411838-21needreboot19:58:000
360412993326,5cyclictest487256-21kworker/u8:2+flush-8:022:13:080
360412993324,5cyclictest0-21swapper/019:15:030
36041299332,30cyclictest0-21swapper/019:43:080
360412993315,16cyclictest561358-21unixbench_singl22:28:080
360412993314,17cyclictest576151-21sort22:43:080
360416993224,6cyclictest596640-21apt-get23:07:512
360416993211,10cyclictest640747-21ntpq23:48:032
360415993228,3cyclictest422786-21kworker/u8:3+flush-8:020:33:081
360415993227,3cyclictest420227-21kthreadcore20:08:001
360415993226,5cyclictest651539-21kworker/u8:4+flush-8:000:28:051
360415993226,5cyclictest651538-21kworker/u8:1+flush-8:000:37:531
360415993226,5cyclictest387009-21kworker/u8:0+flush-8:019:38:031
360415993220,9cyclictest275-21systemd-journal20:28:381
360415993220,9cyclictest275-21systemd-journal20:28:381
36041299326,20cyclictest521rcuop/321:43:030
36041299323,25cyclictest462539-21apt-get20:52:510
36041299323,24cyclictest0-21swapper/023:57:500
36041299323,23cyclictest360776-21apt-get19:12:520
360412993216,14cyclictest526699-21unixbench_singl21:53:110
360412993216,14cyclictest0-21swapper/000:03:110
36041299321,4cyclictest591724-21apt-get23:02:480
360412993212,18cyclictest600903-21unixbench_multi23:08:110
360416993117,11cyclictest562011-21/usr/sbin/munin22:32:522
360416993111,10cyclictest590598-21ntpq22:58:072
360415993125,4cyclictest387009-21kworker/u8:0+flush-8:021:23:051
36041299317,22cyclictest459520-21cut20:47:580
36041299311,9cyclictest531169-21ntpq21:58:040
360412993115,14cyclictest551648-21idleruntime-cro22:20:000
360412993114,15cyclictest0-21swapper/023:53:110
36041299311,25cyclictest690146-21taskset00:38:040
360416993018,10cyclictest504749-21/usr/sbin/munin21:33:002
360416993017,9cyclictest647196-21/usr/sbin/munin23:57:552
360416993017,11cyclictest685339-21/usr/sbin/munin00:33:072
360416993017,10cyclictest482551-21/usr/sbin/munin21:12:522
360416993010,10cyclictest481430-21ntpq21:08:042
360415993025,3cyclictest651539-21kworker/u8:4+flush-8:000:12:541
360415993023,5cyclictest675857-21unixbench_singl00:23:111
360415993023,5cyclictest451913-21grep20:38:091
36041299303,24cyclictest610548-21ntpq23:18:030
36041299302,24cyclictest521186-21ntpq21:48:030
36041299302,1cyclictest0-21swapper/023:48:010
360412993018,10cyclictest452647-21apt-get20:42:480
36041299301,26cyclictest580691-21ntpq22:48:030
36041699299,5cyclictest1-21systemd21:30:002
36041699299,10cyclictest655521-21ntpq00:03:032
360416992923,4cyclictest388496-21fschecks_time19:37:532
360416992917,9cyclictest641942-21/usr/sbin/munin23:52:542
360416992917,9cyclictest543080-21/usr/sbin/munin22:13:002
360416992916,11cyclictest441827-21/usr/sbin/munin20:28:102
360416992916,11cyclictest441827-21/usr/sbin/munin20:28:092
360416992916,11cyclictest367897-21/usr/sbin/munin19:17:502
360416992916,10cyclictest547629-21/usr/sbin/munin22:17:562
360416992914,12cyclictest487352-21chrt21:16:202
360416992911,4cyclictest575765-21ntpq22:43:022
360415992925,3cyclictest317305-21kworker/u8:3+flush-8:019:33:031
360415992922,5cyclictest571719-21kworker/u8:2+flush-8:023:58:031
360415992922,4cyclictest0-21swapper/123:17:511
36041299292,24cyclictest627015-21apt-get23:37:510
36041299292,23cyclictest360414-21latency_hist19:10:000
36041299290,27cyclictest681471-21aten2_r5power_c00:32:510
360416992824,2cyclictest552148-21aten2_r5power_c22:22:512
360416992821,5cyclictest451320-21grep20:38:012
360416992818,7cyclictest576950-21/usr/sbin/munin22:47:512
360416992817,9cyclictest665484-21/usr/sbin/munin00:13:032
360416992817,9cyclictest476290-21/usr/sbin/munin21:03:072
360416992816,8cyclictest616216-21apt-get23:25:022
360416992813,12cyclictest467352-21seq20:56:362
360416992810,4cyclictest516293-21ntpq21:43:032
360415992824,2cyclictest507057-21cron21:35:001
360415992823,3cyclictest398519-21cpuspeed_turbos19:47:541
360415992822,5cyclictest621639-21kworker/u8:3+events_unbound23:33:091
360416992724,2cyclictest571262-21unixbench_singl22:38:102
360416992723,2cyclictest452731-21aten2_r5power_p20:42:522
360416992721,4cyclictest365312-21idleruntime-cro19:15:002
360416992719,7cyclictest631306-21unixbench_singl23:38:102
360416992718,7cyclictest422295-21/usr/sbin/munin20:08:112
360416992717,8cyclictest496316-21/usr/sbin/munin21:23:072
360416992717,7cyclictest666813-21/usr/sbin/munin00:17:512
360416992717,7cyclictest417358-21/usr/sbin/munin20:03:102
360416992717,7cyclictest378587-21/usr/sbin/munin19:27:552
360416992717,7cyclictest378587-21/usr/sbin/munin19:27:552
360416992716,9cyclictest461541-21/usr/sbin/munin20:48:072
360416992716,9cyclictest383417-21/usr/sbin/munin19:32:522
360415992721,4cyclictest571719-21kworker/u8:2+flush-8:023:38:001
360416992621,3cyclictest398605-21fschecks_time19:47:552
360416992617,7cyclictest462980-21/usr/sbin/munin20:52:562
360416992617,7cyclictest433204-21fschecks_count20:22:552
360416992617,6cyclictest412167-21/usr/sbin/munin19:58:072
360416992615,9cyclictest522378-21/usr/sbin/munin21:52:512
360416992614,9cyclictest581777-21chrt22:52:222
360416992614,9cyclictest536278-21/usr/sbin/munin22:03:072
360416992614,7cyclictest656736-21apt-get00:07:522
360416992612,11cyclictest591487-21chrt23:00:042
360416992612,11cyclictest442674-21chrt20:32:032
360415992621,4cyclictest20823-21cron19:55:001
360416992518,5cyclictest675888-21/usr/sbin/munin00:23:112
360416992517,6cyclictest606994-21/usr/sbin/munin23:17:522
360416992516,7cyclictest541041-21/usr/sbin/munin22:08:052
360416992515,8cyclictest427052-21/usr/sbin/munin20:13:072
360416992515,7cyclictest686344-21/usr/sbin/munin00:37:492
360416992514,9cyclictest521433-21/usr/sbin/munin21:48:092
360416992513,9cyclictest631760-21taskset23:40:092
360416992513,9cyclictest556819-21seq22:25:072
360416992511,12cyclictest531799-21chrt21:59:502
360416992413,9cyclictest508121-21sed21:37:562
360416992412,9cyclictest601424-21chrt23:10:552
360416992412,9cyclictest427851-21chrt20:16:302
360416992412,9cyclictest403024-21chrt19:50:342
360416992411,10cyclictest676349-21chrt00:25:582
360415992419,3cyclictest592649-21sed23:02:571
360415992418,4cyclictest621639-21kworker/u8:3+flush-8:000:03:001
360416992313,8cyclictest373017-21seq19:21:402
360415992319,2cyclictest0-21swapper/100:07:501
3556452239,9sleep10-21swapper/119:05:401
360416992212,7cyclictest360523-21sed19:10:012
360415992014,4cyclictest0-21swapper/121:52:561
3600882193,9sleep30-21swapper/319:08:113
6814472152,9sleep30-21swapper/300:32:503
6661852132,6sleep30-21swapper/300:15:013
6545562132,6sleep30-21swapper/300:03:013
6381142133,7sleep30-21swapper/323:47:573
6310452133,6sleep30-21swapper/323:38:113
6155352132,7sleep30-21swapper/323:23:063
5764582132,6sleep30-21swapper/322:45:013
5036132134,5sleep30-21swapper/321:33:013
4268072132,7sleep30-21swapper/320:13:033
4175632131,6sleep30-21swapper/320:05:023
3605942135,5sleep30-21swapper/319:10:483
6863642122,6sleep30-21swapper/300:37:523
6751772122,6sleep30-21swapper/300:23:033
6639292122,6sleep30-21swapper/300:13:003
6564452122,5sleep30-21swapper/300:05:023
6413952122,5sleep30-21swapper/323:50:013
6071662122,6sleep30-21swapper/323:17:553
6058212122,6sleep30-21swapper/323:13:103
5980132122,6sleep30-21swapper/323:08:003
5915302122,6sleep30-21swapper/323:00:383
5609122122,6sleep30-21swapper/322:28:043
5454842122,6sleep30-21swapper/322:13:043
5368062122,5sleep30-21swapper/322:05:013
5277282122,6sleep30-21swapper/321:57:553
5223572122,6sleep30-21swapper/321:52:383
5178132122,6sleep30-21swapper/321:47:573
5111442122,6sleep30-21swapper/321:38:013
4970352122,5sleep30-21swapper/321:25:013
4872502122,5sleep30-21swapper/321:15:023
4816692122,6sleep30-21swapper/321:08:083
4719872122,5sleep30-21swapper/321:00:023
4678642122,6sleep30-21swapper/320:57:573
4635852122,6sleep30-21swapper/320:52:593
4612922122,6sleep30-21swapper/320:48:023
4421262122,6sleep30-21swapper/320:28:123
4421262122,6sleep30-21swapper/320:28:123
4325322122,5sleep30-21swapper/320:20:013
4181032122,6sleep30-21swapper/320:07:543
4118982122,6sleep30-21swapper/319:58:033
4041672122,6sleep30-21swapper/319:53:133
3975712122,6sleep30-21swapper/319:43:103
3773602122,6sleep30-21swapper/319:23:123
3675812123,6sleep30-21swapper/319:15:063
6471702111,6sleep30-21swapper/323:57:563
6317072112,5sleep30-21swapper/323:40:023
6222272112,5sleep30-21swapper/323:32:563
6168222112,5sleep30-21swapper/323:27:023
5911962112,5sleep30-21swapper/322:59:383
5817112112,5sleep30-21swapper/322:51:313
5678212112,5sleep30-21swapper/322:37:593
5619212112,6sleep30-21swapper/322:32:013
5519212113,5sleep30-21swapper/322:20:383
5476462112,5sleep30-21swapper/322:17:573
5320882112,6sleep30-21swapper/322:00:163
5120702112,5sleep30-21swapper/321:40:023
4826482112,5sleep30-21swapper/321:12:533
4526192112,5sleep30-21swapper/320:42:483
4476962112,5sleep30-21swapper/320:37:513
4430072112,6sleep30-21swapper/320:32:533
4288662112,5sleep30-21swapper/320:17:573
3982932113,5sleep30-21swapper/319:47:533
3926272110,6sleep30-21swapper/319:40:013
3829742112,5sleep30-21swapper/319:30:063
3775842112,5sleep30-21swapper/319:25:013
3775842112,5sleep30-21swapper/319:25:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional