You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 02:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Wed Dec 03, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
346518822020,2sleep3586-21kworker/3:2+events19:41:163
3430495997450,8cyclictest3720336-21cp00:00:012
3430487997470,3cyclictest3610411-21kworker/u8:2+events_unbound23:01:250
3430487996864,3cyclictest3729817-21kworker/u8:0+events_unbound00:31:240
3430491996661,4cyclictest3420780-21kworker/u8:0+events_unbound19:11:251
3430491996661,4cyclictest3420780-21kworker/u8:0+events_unbound19:11:241
3430491996359,2cyclictest3551041-21kworker/u8:1+flush-8:021:21:241
3430491996359,2cyclictest3551041-21kworker/u8:1+flush-8:021:21:241
35774522620,3sleep3586-21kworker/3:2+events21:36:123
3430487996253,3cyclictest3690381-21kworker/u8:3+events_unbound23:56:250
3430487996157,3cyclictest3590510-21kworker/u8:1+events_unbound22:06:220
3430487996056,3cyclictest3453228-21kworker/u8:2+events_unbound20:36:260
3430491995850,6cyclictest3684915-21kworker/u8:2+flush-8:023:46:191
3430491995748,3cyclictest3430358-21kworker/u8:1+events_unbound19:46:091
3430487995753,3cyclictest3580613-21kworker/u8:0+events_unbound21:41:240
3430487995752,3cyclictest3610411-21kworker/u8:2+events_unbound22:31:210
3430487995749,3cyclictest3453228-21kworker/u8:2+events_unbound19:56:120
3430491995650,4cyclictest3453228-21kworker/u8:2+events_unbound20:21:131
3430487995652,3cyclictest3453228-21kworker/u8:2+events_unbound19:41:240
3430491995550,4cyclictest3690381-21kworker/u8:3+flush-8:000:36:201
3430491995550,4cyclictest3430358-21kworker/u8:1+events_unbound19:31:181
3430487995552,2cyclictest3675527-21kworker/u8:0+events_unbound23:36:260
3430487995551,3cyclictest3670088-21kworker/u8:3+events_unbound23:11:230
3430487995550,4cyclictest3719755-21kworker/u8:1+events_unbound00:06:230
3430491995449,3cyclictest3516433-21kworker/u8:3+events_unbound22:11:091
3430491995444,8cyclictest282-21systemd-journal20:30:001
3430487995450,3cyclictest3684915-21kworker/u8:2+events_unbound23:46:220
3430487995444,8cyclictest3437914-21diskstats19:16:100
3430487995444,8cyclictest3437914-21diskstats19:16:100
3430491995348,4cyclictest3580613-21kworker/u8:0+events_unbound21:36:271
3430491995347,4cyclictest3690381-21kworker/u8:3+flush-8:023:41:201
3430491995346,5cyclictest3641393-21kworker/u8:3+flush-8:022:51:131
3430487995349,3cyclictest3729817-21kworker/u8:0+events_unbound00:11:200
3430487995349,3cyclictest3625515-21kworker/u8:0+events_unbound22:41:170
3430487995349,3cyclictest3516433-21kworker/u8:3+events_unbound22:11:230
3430487995349,3cyclictest3501948-21kworker/u8:0+events_unbound20:31:220
3430487995348,4cyclictest3516433-21kworker/u8:3+events_unbound21:56:220
3430491995248,3cyclictest3420780-21kworker/u8:0+events_unbound19:21:111
3430491995247,3cyclictest3516433-21kworker/u8:3+events_unbound20:36:191
3430491995247,3cyclictest3501948-21kworker/u8:0+events_unbound20:31:081
3430491995246,4cyclictest3641393-21kworker/u8:3+flush-8:022:56:181
3430491995245,5cyclictest3655289-21kworker/u8:0+flush-8:023:06:201
3430487995248,3cyclictest3719755-21kworker/u8:1+events_unbound00:01:120
3430487995248,3cyclictest3641393-21kworker/u8:3+events_unbound22:51:180
3430487995248,3cyclictest3580613-21kworker/u8:0+events_unbound21:51:250
3430487995248,3cyclictest3430358-21kworker/u8:1+events_unbound19:51:240
3430487995247,3cyclictest3430358-21kworker/u8:1+events_unbound19:46:210
3430487995245,6cyclictest3420780-21kworker/u8:0+events_unbound19:21:100
3430487995243,3cyclictest3521501-21kworker/u8:0+events_unbound21:01:240
36995602510,2sleep10-21swapper/123:36:231
3430491995145,5cyclictest3487122-21kworker/u8:3+flush-8:020:16:211
3430487995147,3cyclictest3516433-21kworker/u8:3+events_unbound22:26:260
3430487995147,3cyclictest3430358-21kworker/u8:1+events_unbound20:06:080
3430487995146,4cyclictest3516433-21kworker/u8:3+events_unbound20:46:100
3430487995145,4cyclictest3729817-21kworker/u8:0+events_unbound00:16:100
3430491995046,3cyclictest3516433-21kworker/u8:3+events_unbound20:56:191
3430491995046,3cyclictest3430358-21kworker/u8:1+flush-8:019:26:191
3430491995044,5cyclictest3690381-21kworker/u8:3+flush-8:023:56:201
3430491995044,4cyclictest3590510-21kworker/u8:1+events_unbound22:06:201
3430491995039,4cyclictest3516433-21kworker/u8:3+flush-8:022:16:211
3430487995046,3cyclictest3516433-21kworker/u8:3+events_unbound20:41:220
3430487995045,4cyclictest3660702-21kworker/u8:1+events_unbound23:26:190
3430487995045,4cyclictest3575316-21kworker/u8:2+events_unbound21:36:230
3430491994943,5cyclictest3516433-21kworker/u8:3+flush-8:022:01:121
3430491994943,4cyclictest3690381-21kworker/u8:3+flush-8:023:31:111
3430491994938,3cyclictest3575316-21kworker/u8:2+events_unbound21:46:091
3430487994945,3cyclictest3690381-21kworker/u8:3+events_unbound00:26:130
3430487994945,3cyclictest3690381-21kworker/u8:3+events_unbound00:26:130
3430487994945,3cyclictest3690381-21kworker/u8:3+events_unbound00:21:070
3430487994944,4cyclictest3675527-21kworker/u8:0+flush-8:023:16:170
3430487994943,4cyclictest3590510-21kworker/u8:1+events_unbound22:01:220
3430491994845,2cyclictest3516433-21kworker/u8:3+events_unbound21:06:191
3430491994843,4cyclictest3453228-21kworker/u8:2+flush-8:020:01:181
3430491994842,5cyclictest3690381-21kworker/u8:3+flush-8:000:26:261
3430491994842,5cyclictest3690381-21kworker/u8:3+flush-8:000:26:261
3430491994842,5cyclictest3516433-21kworker/u8:3+flush-8:020:51:211
3430491994841,5cyclictest3610411-21kworker/u8:2+flush-8:022:21:191
3430487994844,3cyclictest3641393-21kworker/u8:3+events_unbound22:46:110
3430487994844,3cyclictest3516433-21kworker/u8:3+events_unbound21:06:080
3430487994844,3cyclictest3430358-21kworker/u8:1+events_unbound19:36:210
3430487994843,4cyclictest3690381-21kworker/u8:3+events_unbound00:36:230
3430487994843,4cyclictest3430358-21kworker/u8:1+events_unbound19:26:230
3430487994839,7cyclictest3655841-21diskstats22:56:090
3430491994742,4cyclictest3670088-21kworker/u8:3+events_unbound23:11:201
3430491994742,4cyclictest3610411-21kworker/u8:2+flush-8:022:31:191
3430491994742,3cyclictest3551041-21kworker/u8:1+flush-8:021:16:231
3430491994742,3cyclictest3430358-21kworker/u8:1+events_unbound20:11:271
3430491994741,5cyclictest3453228-21kworker/u8:2+flush-8:019:56:201
3430491994740,5cyclictest3675527-21kworker/u8:0+flush-8:023:51:211
3430491994740,5cyclictest3675527-21kworker/u8:0+flush-8:023:51:211
3430487994743,3cyclictest3675527-21kworker/u8:0+events_unbound23:51:130
3430487994743,3cyclictest3675527-21kworker/u8:0+events_unbound23:51:130
3430487994743,3cyclictest3561482-21kworker/u8:0+events_unbound21:26:240
3430487994743,3cyclictest3420780-21kworker/u8:0+events_unbound19:11:270
3430487994743,3cyclictest3420780-21kworker/u8:0+events_unbound19:11:270
3430487994742,4cyclictest3670088-21kworker/u8:3+events_unbound23:21:260
3430487994742,4cyclictest3453228-21kworker/u8:2+events_unbound20:26:130
3430491994642,3cyclictest3675527-21kworker/u8:0+events_unbound23:16:181
3430491994641,4cyclictest3430358-21kworker/u8:1+flush-8:019:16:211
3430491994641,4cyclictest3430358-21kworker/u8:1+flush-8:019:16:211
3430491994641,4cyclictest3430358-21kworker/u8:1+events_unbound19:51:181
3430491994641,3cyclictest3719755-21kworker/u8:1+flush-8:000:01:071
3430491994640,5cyclictest3610411-21kworker/u8:2+flush-8:022:26:191
3430491994636,3cyclictest3729817-21kworker/u8:0+events_unbound00:16:241
3430487994642,3cyclictest3610411-21kworker/u8:2+events_unbound22:21:070
3430487994634,10cyclictest282-21systemd-journal20:25:000
3430487994542,2cyclictest3516433-21kworker/u8:3+events_unbound20:51:090
3430487994541,3cyclictest3690381-21kworker/u8:3+events_unbound23:41:070
3430487994541,3cyclictest3625515-21kworker/u8:0+events_unbound22:36:170
3430487994541,3cyclictest3551041-21kworker/u8:1+events_unbound21:21:290
3430487994541,3cyclictest3551041-21kworker/u8:1+events_unbound21:21:290
3430487994535,8cyclictest3556854-21diskstats21:16:080
3430495994441,2cyclictest3634971-21kworker/u8:1+events_unbound22:55:002
3430491994440,3cyclictest3641393-21kworker/u8:3+flush-8:022:41:221
3430491994438,4cyclictest3660702-21kworker/u8:1+events_unbound23:21:111
3430487994439,4cyclictest3453228-21kworker/u8:2+events_unbound20:01:170
3430491994339,3cyclictest3729817-21kworker/u8:0+events_unbound00:11:191
3430491994339,3cyclictest3625515-21kworker/u8:0+flush-8:022:36:201
3430491994337,4cyclictest3660805-21apt-get23:01:061
3430487994339,3cyclictest3551041-21kworker/u8:1+events_unbound21:11:070
3430487994337,4cyclictest3516433-21kworker/u8:3+events_unbound20:56:230
3430491994237,4cyclictest3516433-21kworker/u8:3+flush-8:020:46:201
3430491994236,4cyclictest3453228-21kworker/u8:2+events_unbound19:41:221
3430491994235,5cyclictest3641393-21kworker/u8:3+flush-8:022:46:171
3430487994238,3cyclictest3575316-21kworker/u8:2+events_unbound21:46:070
3430487994238,3cyclictest3430358-21kworker/u8:1+events_unbound19:31:100
3430491994137,3cyclictest3561482-21kworker/u8:0+flush-8:021:26:251
3430491994137,3cyclictest3521501-21kworker/u8:0+flush-8:021:01:201
3430491994136,4cyclictest3580613-21kworker/u8:0+flush-8:021:41:251
3430491994136,4cyclictest3551041-21kworker/u8:1+flush-8:021:11:201
3430491994135,5cyclictest3729817-21kworker/u8:0+flush-8:000:31:191
3430491994131,8cyclictest282-21systemd-journal21:56:121
3430487994138,2cyclictest3516433-21kworker/u8:3+events_unbound21:31:110
3430487994137,3cyclictest3655289-21kworker/u8:0+events_unbound23:06:270
3430487994035,4cyclictest3516433-21kworker/u8:3+events_unbound22:18:500
3430491993833,4cyclictest3719755-21kworker/u8:1+events_unbound00:06:191
3430491993832,5cyclictest3660702-21kworker/u8:1+flush-8:023:26:181
3430491993732,3cyclictest3516433-21kworker/u8:3+flush-8:021:31:171
3430491993732,3cyclictest3430358-21kworker/u8:1+events_unbound19:36:121
3430487993733,3cyclictest3690381-21kworker/u8:3+events_unbound23:31:110
3430495993630,4cyclictest282-21systemd-journal21:47:122
3430487993620,9cyclictest3487122-21kworker/u8:3-events_unbound20:17:100
3430495993325,6cyclictest282-21systemd-journal21:35:012
3430495993323,8cyclictest282-21systemd-journal22:07:122
3430495993321,9cyclictest3521467-21cat20:40:002
3430491993329,3cyclictest3690381-21kworker/u8:3+events_unbound00:21:131
3430495993221,8cyclictest3600819-21awk22:00:002
3430487993228,3cyclictest3487122-21kworker/u8:3+events_unbound20:12:120
3430495993125,4cyclictest282-21systemd-journal21:37:122
3430495993124,5cyclictest3750055-21base6400:30:002
3430495993124,5cyclictest3750055-21base6400:30:002
3430491993127,3cyclictest3430358-21kworker/u8:1+flush-8:020:06:221
3430491993126,4cyclictest3516433-21kworker/u8:3+events_unbound21:51:111
3430495993026,3cyclictest3670033-21unixbench_singl23:06:262
3430495992919,8cyclictest3481058-21ntpq19:56:182
3430495992919,8cyclictest282-21systemd-journal20:19:122
3430491992925,3cyclictest3453228-21kworker/u8:2+events_unbound20:41:161
3430495992824,3cyclictest3571128-21idleruntime-cro21:30:002
3430495992824,2cyclictest3611110-21aten2_r5power_p22:11:072
3430495992823,4cyclictest3634977-21unixbench_multi22:31:242
3430495992819,7cyclictest3760250-21cat00:40:002
3430495992818,8cyclictest3695210-21cat23:34:592
3430495992621,3cyclictest3674976-21unixbench_singl23:11:262
36606162250,2sleep3471rcuc/323:00:013
3430495992522,2cyclictest3492407-21fschecks_count20:11:102
3430495992520,4cyclictest3501948-21kworker/u8:0+flush-8:020:21:232
3430495992520,3cyclictest3431046-21fschecks_time19:11:102
3430495992520,3cyclictest3431046-21fschecks_time19:11:102
3430495992518,5cyclictest3685208-21awk23:25:002
3430495992421,2cyclictest3735529-21aten2_r5power_e00:16:062
3430495992420,2cyclictest3724940-21unixbench_singl00:01:252
3430495992420,2cyclictest3705688-21aten2_r5power_p23:46:072
3430495992417,5cyclictest282-21systemd-journal21:15:012
3430495992415,7cyclictest3536287-21apt-get20:55:032
3430495992320,2cyclictest3695960-21fschecks_count23:36:102
3430495992318,3cyclictest3471681-21unixbench_multi19:46:242
342664222312,7sleep13426678-21kthreadcore19:06:151
3430495992219,2cyclictest3725869-21fschecks_count00:06:102
3430495992218,2cyclictest3521841-21aten2_r5power_c20:41:062
3430495992217,3cyclictest0-21swapper/221:10:002
34291702222,13sleep03429363-21grep19:06:180
343049599212,18cyclictest0-21swapper/219:16:162
343049599212,18cyclictest0-21swapper/219:16:162
3430495992118,2cyclictest3616249-21fschecks_count22:16:102
3430495992118,2cyclictest3507261-21fschecks_count20:26:122
3430495992117,2cyclictest3591588-21fschecks_time21:51:102
3430495992113,6cyclictest405-21dbus-daemon19:45:002
3430495992112,7cyclictest3474174-21seq19:51:162
3430495992017,2cyclictest3487560-21fschecks_time20:06:102
3430495992016,2cyclictest3565811-21unixbench_multi21:21:262
3430495992016,2cyclictest3565811-21unixbench_multi21:21:252
3430495992016,2cyclictest0-21swapper/223:45:002
3430495992016,2cyclictest0-21swapper/222:25:002
343049599192,12cyclictest3551648-21apt-get21:11:072
3430495991916,2cyclictest3680284-21sendmail23:20:002
3430495991916,2cyclictest3546372-21sh21:05:002
3430495991915,2cyclictest0-21swapper/220:46:092
3430495991913,5cyclictest3640761-21aten2_r5power_e22:41:062
34294332194,8sleep20-21swapper/219:06:202
343049599189,6cyclictest3659253-21cat22:56:172
343049599189,6cyclictest3462113-21awk19:40:012
343049599189,2cyclictest0-21swapper/222:46:082
343049599188,2cyclictest0-21swapper/222:26:182
343049599185,9cyclictest1-21systemd19:25:002
343049599182,12cyclictest3660217-21apt-get23:00:032
3430495991814,2cyclictest3601188-21aten2_r5power_c22:01:062
3430495991814,2cyclictest3581470-21aten2_r5power_v21:41:092
3430495991814,2cyclictest0-21swapper/220:55:002
3430495991810,6cyclictest3731159-21sed00:11:102
34256792183,8sleep30-21swapper/319:05:243
3430495991713,2cyclictest3710738-21cpuspeed_turbos23:51:102
3430495991713,2cyclictest3710738-21cpuspeed_turbos23:51:102
3430495991713,2cyclictest0-21swapper/223:26:062
3430495991710,5cyclictest3750867-21fschecks_time00:31:092
343049599168,6cyclictest3451841-21unixbench_singl19:26:252
343049599162,13cyclictest0-21swapper/219:31:362
3430495991613,2cyclictest0-21swapper/222:37:202
3430495991613,2cyclictest0-21swapper/200:20:452
3430495991612,2cyclictest0-21swapper/220:31:242
35516512152,8sleep30-21swapper/321:11:063
343049599157,6cyclictest3483254-21tr20:01:142
36344482145,6sleep30-21swapper/322:31:163
35371282143,8sleep30-21swapper/320:56:093
37151012132,7sleep30-21swapper/323:55:003
37151012132,7sleep30-21swapper/323:54:593
36761002132,7sleep30-21swapper/323:16:113
36703652132,6sleep30-21swapper/323:10:013
35915142133,6sleep30-21swapper/321:51:113
35612792132,6sleep30-21swapper/321:20:013
35612792132,6sleep30-21swapper/321:20:013
34430832133,6sleep30-21swapper/319:21:143
37551962122,5sleep30-21swapper/300:35:013
37397992122,6sleep30-21swapper/300:16:273
37308832122,6sleep30-21swapper/300:11:093
37259132122,6sleep30-21swapper/300:06:113
37010482122,6sleep30-21swapper/323:41:113
36952192122,5sleep30-21swapper/323:35:013
36943972122,6sleep30-21swapper/323:31:243
36855652122,6sleep30-21swapper/323:26:063
36842962122,6sleep30-21swapper/323:21:173
36702752122,6sleep30-21swapper/323:09:013
36506832122,6sleep30-21swapper/322:51:073
36411642122,6sleep30-21swapper/322:41:123
36217212122,6sleep30-21swapper/322:21:133
36063352121,7sleep30-21swapper/322:06:083
36002822123,6sleep30-21swapper/321:56:253
35817292122,6sleep30-21swapper/321:41:123
35471982122,6sleep30-21swapper/321:06:123
35460332122,6sleep30-21swapper/321:01:273
35307462122,6sleep30-21swapper/320:46:263
35170862122,6sleep30-21swapper/320:36:113
35118752122,6sleep30-21swapper/320:31:053
35017002123,5sleep30-21swapper/320:20:013
34962102122,6sleep30-21swapper/320:11:273
34912192122,6sleep30-21swapper/320:06:253
34761032122,6sleep30-21swapper/319:51:203
34567652122,6sleep30-21swapper/319:31:263
34310662122,6sleep30-21swapper/319:11:113
34310662122,6sleep30-21swapper/319:11:113
37545422112,5sleep30-21swapper/300:31:243
37458712112,5sleep30-21swapper/300:26:143
37458712112,5sleep30-21swapper/300:26:143
37409042112,5sleep30-21swapper/300:21:123
37245082112,5sleep30-21swapper/300:01:223
37198472112,6sleep30-21swapper/323:57:533
37058402112,5sleep30-21swapper/323:46:093
36644222112,5sleep30-21swapper/323:01:173
36456302113,5sleep30-21swapper/322:45:293
36402012112,6sleep30-21swapper/322:38:193
36302142112,6sleep30-21swapper/322:27:073
36162692112,5sleep30-21swapper/322:16:113
36107112112,5sleep30-21swapper/322:10:003
36055122112,5sleep30-21swapper/322:01:593
35865682112,6sleep30-21swapper/321:46:103
35713592112,5sleep30-21swapper/321:30:123
35702722112,5sleep30-21swapper/321:26:233
35565102112,5sleep30-21swapper/321:15:113
35320112112,5sleep30-21swapper/320:51:113
35220522112,5sleep30-21swapper/320:41:083
35068712112,6sleep30-21swapper/320:25:113
34974542112,5sleep30-21swapper/320:16:113
34826492112,5sleep30-21swapper/320:01:103
34780432112,5sleep30-21swapper/319:56:143
34717952112,6sleep30-21swapper/319:47:073
34574712112,5sleep30-21swapper/319:36:073
34475012112,5sleep30-21swapper/319:25:073
34376972112,5sleep30-21swapper/319:16:093
34376972112,5sleep30-21swapper/319:16:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional