You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 09:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Thu Nov 20, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20991862900,2chrt586-21kworker/3:2-events20:26:223
2055141997871,5cyclictest2166201-21kworker/u8:1+flush-8:021:46:291
2055141997066,3cyclictest2121752-21kworker/u8:0+events_unbound21:31:291
2055139996858,7cyclictest2234699-21ntpq22:41:270
2055141996762,4cyclictest2131378-21kworker/u8:2+events_unbound21:06:301
2055141996460,3cyclictest2285214-21kworker/u8:2+events_unbound00:31:291
2055141996458,5cyclictest2076750-21kworker/u8:3+flush-8:020:41:321
2055141996458,4cyclictest2121752-21kworker/u8:0+events_unbound20:56:131
2055139996352,8cyclictest2334355-21ntpq00:21:240
2055141996255,4cyclictest2121752-21kworker/u8:0+flush-8:021:51:311
2055141996155,5cyclictest2298342-21kworker/u8:1+events_unbound00:06:281
2055141996155,4cyclictest2181353-21kworker/u8:2+events_unbound22:11:261
2055141996155,4cyclictest2121752-21kworker/u8:0+events_unbound21:41:271
2055145996053,5cyclictest2250888-21cat23:00:012
2055141996052,6cyclictest2076750-21kworker/u8:3+flush-8:020:46:261
2055141995954,4cyclictest2275825-21kworker/u8:3+events_unbound23:26:271
2055141995954,3cyclictest2061544-21kworker/u8:1+events_unbound21:11:281
2055141995853,4cyclictest2240552-21kworker/u8:2+events_unbound23:21:301
2055141995853,4cyclictest2076750-21kworker/u8:3+events_unbound20:11:271
2055141995851,5cyclictest2175901-21kworker/u8:3+flush-8:022:01:271
2055141995850,6cyclictest2334751-21kworker/u8:3+flush-8:000:26:261
2055141995750,6cyclictest2166201-21kworker/u8:1+events_unbound21:36:241
2055141995750,5cyclictest2298342-21kworker/u8:1+flush-8:000:11:271
2055145995651,3cyclictest2320177-21gpgv00:10:012
2055141995652,3cyclictest2220989-21kworker/u8:0+flush-8:022:51:311
2055139995645,8cyclictest2116358-21apt-get20:45:000
2055141995550,4cyclictest2215980-21kworker/u8:3+flush-8:022:41:241
2055141995549,5cyclictest2086878-21kworker/u8:2+events_unbound20:31:351
2055141995447,5cyclictest2156640-21kworker/u8:3+flush-8:021:26:301
2055141995447,5cyclictest2121752-21kworker/u8:0+events_unbound21:56:261
2055139995443,8cyclictest2224843-21ntpq22:31:260
2055141995349,3cyclictest2061544-21kworker/u8:1+events_unbound20:16:171
2055141995348,3cyclictest2061544-21kworker/u8:1+events_unbound20:21:221
2055141995346,5cyclictest2285214-21kworker/u8:2+events_unbound00:01:291
2055141995346,5cyclictest2195736-21kworker/u8:1+flush-8:023:31:311
2055141995346,5cyclictest2195736-21kworker/u8:1+flush-8:022:31:311
2055141995346,5cyclictest2181353-21kworker/u8:2+flush-8:022:21:261
2055141995245,5cyclictest2240552-21kworker/u8:2+events_unbound23:16:161
2055139995241,8cyclictest2244581-21ntpq22:51:230
2055141995144,6cyclictest2121752-21kworker/u8:0+events_unbound21:16:261
2055139995142,6cyclictest2066416-21cp19:55:010
2055145995041,6cyclictest2058828-21apt-get19:25:002
2055141995045,3cyclictest2220989-21kworker/u8:0+events_unbound23:56:301
2055141995045,3cyclictest2220989-21kworker/u8:0+events_unbound23:56:301
2055141995044,4cyclictest2285221-21kworker/u8:4+events_unbound23:41:121
2055141995044,4cyclictest2061544-21kworker/u8:1+flush-8:020:26:211
2055141995044,4cyclictest1703226-21kworker/u8:0+events_unbound20:01:391
2055141995039,9cyclictest2329375-21ntpq00:16:251
2055141994945,3cyclictest2220989-21kworker/u8:0+flush-8:022:36:301
2055141994943,4cyclictest2195736-21kworker/u8:1+flush-8:022:26:261
2055141994943,4cyclictest2061544-21kworker/u8:1+flush-8:021:01:251
2055141994941,6cyclictest2298342-21kworker/u8:1+flush-8:023:46:301
2055141994938,8cyclictest2209976-21ntpq22:16:271
2055139994944,3cyclictest2181353-21kworker/u8:2+events_unbound22:06:290
2055141994742,3cyclictest2086878-21kworker/u8:2+events_unbound20:36:281
2055141994741,4cyclictest2076750-21kworker/u8:3+events_unbound20:06:321
2055141994741,4cyclictest2023620-21kworker/u8:3+events_unbound19:56:151
2055141994740,5cyclictest2215980-21kworker/u8:3+events_unbound22:46:141
2055139994743,3cyclictest2220989-21kworker/u8:0+events_unbound22:36:290
2055141994640,4cyclictest2285221-21kworker/u8:4+flush-8:023:36:241
2055141994640,4cyclictest2181353-21kworker/u8:2+flush-8:022:06:271
2055139994642,3cyclictest2195736-21kworker/u8:1+events_unbound22:25:000
2055141994539,4cyclictest2240552-21kworker/u8:2+flush-8:022:56:291
2055141994537,3cyclictest2240552-21kworker/u8:2+flush-8:023:06:261
20859802440,5sleep0161rcu_preempt20:11:340
2055141994433,8cyclictest2254853-21ntpq23:01:301
205457224428,10sleep10-21swapper/119:06:351
2055141994330,9cyclictest2126479-21apt-get20:55:001
2055139994234,6cyclictest2285214-21kworker/u8:2+flush-8:023:51:280
2055139994234,6cyclictest2285214-21kworker/u8:2+flush-8:023:51:280
2055139994129,9cyclictest282-21systemd-journal00:11:270
2055141994034,5cyclictest2240552-21kworker/u8:2+events_unbound23:11:261
2055141994031,7cyclictest2304437-21ntpq23:51:291
2055141994031,7cyclictest2304437-21ntpq23:51:291
2055139994034,4cyclictest2285221-21kworker/u8:4+events_unbound23:36:280
2055139994031,6cyclictest2106446-21latency_hist20:35:010
2055141993928,8cyclictest2058820-21latency_hist19:25:011
2055139993933,4cyclictest1703226-21kworker/u8:0+events_unbound19:15:050
2055141993830,6cyclictest2345440-21apt-get00:36:131
2055139993834,3cyclictest2220989-21kworker/u8:0+events_unbound22:46:280
2055139993732,4cyclictest2121752-21kworker/u8:0+events_unbound22:11:280
2055139993731,4cyclictest2066781-21kworker/u8:2+events_unbound19:56:090
2055139993730,5cyclictest2131378-21kworker/u8:2+flush-8:021:06:300
2055145993625,9cyclictest282-21systemd-journal21:21:272
2055145993625,9cyclictest282-21systemd-journal21:21:272
2055139993632,3cyclictest2061544-21kworker/u8:1+flush-8:020:16:330
2055139993631,4cyclictest2240552-21kworker/u8:2+flush-8:022:56:320
2055139993628,6cyclictest2061544-21kworker/u8:1+flush-8:021:11:270
2055139993627,6cyclictest2062136-21latency_hist19:50:000
2055145993528,5cyclictest282-21systemd-journal19:15:002
2055141993531,3cyclictest2220989-21kworker/u8:0+events_unbound00:21:191
2055139993530,3cyclictest2131378-21kworker/u8:2+flush-8:021:41:330
2055139993529,4cyclictest2220989-21kworker/u8:0+flush-8:000:16:170
2055139993529,4cyclictest2061544-21kworker/u8:1+flush-8:021:16:290
2055139993527,6cyclictest1703226-21kworker/u8:0+flush-8:020:01:400
2055145993423,8cyclictest2066691-21cat19:55:002
2055139993430,3cyclictest2220989-21kworker/u8:0+events_unbound23:06:190
2055139993430,2cyclictest2121516-21cron20:50:000
2055139993428,4cyclictest2195736-21kworker/u8:1+flush-8:022:26:290
2055139993428,4cyclictest2061544-21kworker/u8:1+flush-8:020:21:340
2055139993425,6cyclictest2191043-21latency_hist22:00:010
2055145993325,6cyclictest282-21systemd-journal22:34:592
2055139993327,5cyclictest2220989-21kworker/u8:0+flush-8:023:46:360
2055141993227,3cyclictest2121752-21kworker/u8:0+events_unbound21:21:191
2055141993227,3cyclictest2121752-21kworker/u8:0+events_unbound21:21:191
2055139993225,5cyclictest2279741-21ntpq23:26:290
2055139993224,6cyclictest2121752-21kworker/u8:0+flush-8:021:31:240
20503342323,9sleep20-21swapper/219:05:242
2055139993124,5cyclictest2195736-21kworker/u8:1+flush-8:023:31:250
2055139993121,8cyclictest2130820-21ntpq20:56:240
205513999310,30cyclictest2071545-21kworker/0:0+events21:25:580
2055141993026,3cyclictest1703226-21kworker/u8:0+flush-8:019:40:361
2055139993025,3cyclictest2180815-21unixbench_singl21:46:300
2055141992924,3cyclictest2023620-21kworker/u8:3+flush-8:019:17:201
2055139992925,3cyclictest2298342-21kworker/u8:1+flush-8:000:05:000
2055139992924,3cyclictest2171241-21ls21:40:000
229515522822,3sleep02285221-21kworker/u8:4+flush-8:023:42:000
2055145992822,5cyclictest2096542-21cron20:25:002
2055145992821,5cyclictest282-21systemd-journal22:41:512
205513999285,19cyclictest2071545-21kworker/0:0+events23:10:260
2055139992822,4cyclictest2122592-21tr20:51:210
2055139992819,7cyclictest2335820-21diskstats00:26:160
20952792270,2sleep3471rcuc/320:21:253
2055141992723,3cyclictest1703226-21kworker/u8:0+events_unbound19:39:341
2055141992722,4cyclictest1703226-21kworker/u8:0+events_unbound19:10:431
2055139992722,4cyclictest2220989-21kworker/u8:0+flush-8:000:36:300
205513999272,24cyclictest2071545-21kworker/0:0+events21:21:220
205513999272,24cyclictest2071545-21kworker/0:0+events21:21:220
2055139992722,3cyclictest2266548-21irqrtprio23:16:190
205513999272,23cyclictest2071545-21kworker/0:0+events21:51:180
2055139992719,6cyclictest2131378-21kworker/u8:2+flush-8:021:00:330
21705342260,2sleep3481ktimers/321:36:293
2055145992623,2cyclictest2081456-21cron20:10:002
2055145992622,2cyclictest2101497-21cron20:30:002
2055141992622,3cyclictest2055255-21kworker/u8:1+events_unbound19:25:071
2055141992620,4cyclictest2062128-21cron19:50:001
2055139992622,3cyclictest2061544-21kworker/u8:1+flush-8:020:26:180
2055139992621,4cyclictest2076750-21kworker/u8:3+events_unbound20:06:340
2055139992619,5cyclictest2175901-21kworker/u8:3+events_unbound22:01:280
2055145992522,2cyclictest2196621-21fschecks_count22:06:162
2055145992522,2cyclictest2106815-21aten2_r5power_c20:36:142
2055145992521,3cyclictest2325337-21turbostat.cron00:15:002
2055145992515,7cyclictest2116745-21cat20:45:002
205513999252,22cyclictest2071545-21kworker/0:0+events23:01:180
2055139992519,4cyclictest18912-21cron22:20:000
2055141992420,2cyclictest1-21systemd19:35:011
2055141992417,5cyclictest2066690-21latency_hist19:55:011
205513999242,21cyclictest2310266-21kworker/0:2+events00:06:220
2055139992419,3cyclictest2275232-21unixbench_multi23:21:310
2055145992320,2cyclictest2291056-21cpuspeed_turbos23:41:122
2055145992319,2cyclictest2190705-21unixbench_singl21:56:322
2055145992319,2cyclictest2175800-21grep21:41:292
2055145992319,2cyclictest2087243-21fschecks_count20:16:182
2055145992315,6cyclictest282-21systemd-journal20:35:002
2055145992314,7cyclictest0-21swapper/220:01:352
2055139992319,3cyclictest2061544-21kworker/u8:1+flush-8:020:36:350
2055139992318,4cyclictest1703226-21kworker/u8:0+events_unbound19:15:010
2055139992317,4cyclictest2220989-21kworker/u8:0+flush-8:023:56:270
2055139992317,4cyclictest2220989-21kworker/u8:0+flush-8:023:56:270
2055139992315,6cyclictest2061524-21latency_hist19:45:010
22836592220,2sleep3471rcuc/323:31:223
2055145992219,2cyclictest2310655-21aten2_r5power_e00:01:142
2055145992218,2cyclictest2251068-21aten2_r5power_e23:01:142
2055145992218,2cyclictest0-21swapper/220:15:012
2055145992217,3cyclictest2181692-21tr21:51:162
205514599221,10cyclictest2241694-21irqrtprio22:51:182
205514599212,13cyclictest2116358-21apt-get20:45:032
2055145992118,2cyclictest2236020-21aten2_r5power_v22:46:142
2055145992118,2cyclictest2201612-21fschecks_time22:11:142
2055145992111,3cyclictest0-21swapper/221:46:242
2055139992117,3cyclictest2341098-21sort00:31:230
2055139992111,6cyclictest2059672-21latency_hist19:30:010
205514599202,13cyclictest2146859-21apt-get21:16:132
205514599202,12cyclictest2216090-21apt-get22:26:122
2055145992017,2cyclictest2335887-21fschecks_time00:26:172
2055145992017,2cyclictest2276169-21tr23:26:172
2055145992017,2cyclictest2165901-21unixbench_multi21:31:312
2055145992016,3cyclictest2220989-21kworker/u8:0+flush-8:000:36:222
2055145992016,3cyclictest2142151-21tr21:11:172
2055145992016,2cyclictest2230246-21unixbench_multi22:36:322
2055145992016,2cyclictest0-21swapper/223:10:002
2055145992011,7cyclictest2270555-21sed23:20:002
205513999205,14cyclictest2013462-21kworker/0:1+events19:31:340
2055139992016,2cyclictest0-21swapper/019:40:010
2055139992015,3cyclictest0-21swapper/019:25:010
205514599192,12cyclictest2191399-21apt-get22:01:152
2055145991915,2cyclictest2299765-21/usr/sbin/munin23:46:362
2055145991915,2cyclictest2206286-21aten2_r5power_p22:16:152
2055145991915,2cyclictest0-21swapper/220:55:002
2055145991914,3cyclictest0-21swapper/221:05:002
2055145991910,7cyclictest2261967-21tr23:11:202
205513999192,12cyclictest2055137-21cat19:10:010
20546822193,8sleep30-21swapper/319:06:373
205514599189,7cyclictest2326595-21cut00:16:212
205514599188,8cyclictest282-21systemd-journal23:39:592
2055145991814,2cyclictest2304938-21unixbench_singl23:51:352
2055145991814,2cyclictest2304938-21unixbench_singl23:51:342
2055145991814,2cyclictest0-21swapper/220:56:202
2055145991814,2cyclictest0-21swapper/219:15:022
2055145991814,2cyclictest0-21swapper/200:31:172
2055145991814,2cyclictest0-21swapper/200:21:272
2055145991713,2cyclictest0-21swapper/220:00:002
205514599171,14cyclictest2212078-21sed22:21:192
205514599171,12cyclictest2156756-21apt-get21:26:172
205514599171,12cyclictest2136966-21apt-get21:06:142
2055145991711,2cyclictest0-21swapper/219:44:572
205514599171,11cyclictest2305673-21aten2_r5power_p23:56:142
205514599171,11cyclictest2305673-21aten2_r5power_p23:56:142
205514599168,6cyclictest2271027-21wget23:21:132
205514599162,13cyclictest0-21swapper/219:47:312
205514599162,13cyclictest0-21swapper/219:39:312
2055145991612,2cyclictest0-21swapper/219:30:552
22019712155,6sleep30-21swapper/322:11:173
21509472150,2sleep3471rcuc/321:16:303
205514599152,2cyclictest0-21swapper/221:35:102
2055145991510,3cyclictest0-21swapper/223:31:222
21421182140,3sleep3586-21kworker/3:2-events21:11:173
205514599143,4cyclictest0-21swapper/219:25:142
22756102132,6sleep30-21swapper/323:25:013
22256782132,6sleep30-21swapper/322:35:013
20727352132,7sleep30-21swapper/320:01:233
23442762122,7sleep30-21swapper/300:31:313
23358132122,6sleep30-21swapper/300:26:173
23346322123,5sleep30-21swapper/300:21:313
23088022122,6sleep30-21swapper/323:56:233
23088022122,6sleep30-21swapper/323:56:233
23003582122,6sleep30-21swapper/323:50:013
23003582122,6sleep30-21swapper/323:50:013
22652042122,6sleep30-21swapper/323:11:333
22564652122,6sleep30-21swapper/323:06:183
22504602122,5sleep30-21swapper/323:00:013
22396242122,6sleep30-21swapper/322:46:253
22349812122,6sleep30-21swapper/322:41:353
22303342122,6sleep30-21swapper/322:36:333
22201422122,6sleep30-21swapper/322:26:303
22121032122,6sleep30-21swapper/322:21:203
22003422122,6sleep30-21swapper/322:06:323
21910762122,5sleep30-21swapper/322:00:013
21866852122,6sleep30-21swapper/321:56:163
21814682122,6sleep30-21swapper/321:51:153
21522542122,6sleep30-21swapper/321:21:173
21522542122,6sleep30-21swapper/321:21:173
21275222122,6sleep30-21swapper/320:56:173
21061802122,6sleep30-21swapper/320:31:453
20853302122,6sleep30-21swapper/320:11:253
20777442122,6sleep30-21swapper/320:06:213
20663052120,7sleep30-21swapper/319:55:013
20579482122,6sleep30-21swapper/319:17:013
23494142112,5sleep30-21swapper/300:36:323
23254212112,5sleep30-21swapper/300:15:013
23242112112,5sleep30-21swapper/300:11:243
23169452112,5sleep30-21swapper/300:06:223
23105312112,6sleep30-21swapper/300:00:413
22957772112,5sleep30-21swapper/323:46:203
22925262112,5sleep30-21swapper/323:41:183
22902562112,5sleep30-21swapper/323:37:253
22721232112,5sleep30-21swapper/323:21:203
22658372112,5sleep30-21swapper/323:16:143
22461742111,6sleep30-21swapper/322:56:203
22442762112,5sleep30-21swapper/322:51:223
22062042112,5sleep30-21swapper/322:16:153
21718452112,6sleep30-21swapper/321:41:153
21620292112,5sleep30-21swapper/321:31:163
21569172111,6sleep30-21swapper/321:26:193
21374602111,6sleep30-21swapper/321:06:203
21320122112,5sleep30-21swapper/321:01:153
21217682112,6sleep30-21swapper/320:50:193
21190582112,5sleep30-21swapper/320:46:223
21146142112,5sleep30-21swapper/320:41:243
21067892112,5sleep30-21swapper/320:36:143
20869832112,5sleep30-21swapper/320:16:163
20667942112,6sleep30-21swapper/319:55:173
20619162112,6sleep30-21swapper/319:47:153
20613672112,6sleep30-21swapper/319:43:053
20605192112,6sleep30-21swapper/319:35:153
20592782112,6sleep30-21swapper/319:25:033
20587902113,5sleep30-21swapper/319:24:473
20553492112,5sleep30-21swapper/319:11:273
21763982102,5sleep30-21swapper/321:45:053
20598942102,5sleep30-21swapper/319:30:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional