You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 10:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Jun 26, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2392921080,0sleep10-21swapper/123:30:401
1952729280,8sleep00-21swapper/019:07:250
1938429272,15sleep10-21swapper/119:05:341
1951428976,9sleep20-21swapper/219:07:152
1966628370,9sleep30-21swapper/319:09:143
1982099310,28cyclictest0-21swapper/300:29:083
1981399290,0cyclictest0-21swapper/121:18:241
1981399280,0cyclictest0-21swapper/122:03:331
1981399260,2cyclictest0-21swapper/123:29:481
19813992521,3cyclictest25-21ksoftirqd/123:44:571
1981399242,2cyclictest30491-21kworker/1:200:24:431
1981699230,1cyclictest0-21swapper/223:16:172
1981399237,2cyclictest0-21swapper/100:35:361
1981399230,2cyclictest0-21swapper/122:54:361
1981399230,21cyclictest0-21swapper/121:21:051
1981399230,0cyclictest0-21swapper/122:42:511
27792220,2sleep22761-21gltestperf00:35:172
1982099221,3cyclictest0-21swapper/323:15:233
19816992221,1cyclictest32-21ksoftirqd/223:05:252
19816992221,1cyclictest32-21ksoftirqd/223:05:242
1982099210,2cyclictest0-21swapper/323:00:103
19816992120,1cyclictest0-21swapper/200:29:362
19813992118,2cyclictest0-21swapper/100:05:281
19813992114,4cyclictest0-21swapper/121:29:571
1981399210,2cyclictest0-21swapper/121:32:331
1982099200,1cyclictest0-21swapper/300:11:133
1981399200,19cyclictest0-21swapper/121:55:551
1981399200,17cyclictest0-21swapper/122:48:381
1981399200,15cyclictest0-21swapper/123:02:051
1981699190,18cyclictest19715-21if_err_enp0s31f20:15:192
1981399194,2cyclictest19802-21cyclictest00:25:151
19813991917,1cyclictest6759-21ssh21:45:131
19813991917,1cyclictest6759-21ssh21:45:131
1981399190,1cyclictest0-21swapper/122:15:511
1981399190,1cyclictest0-21swapper/100:32:241
1982099184,4cyclictest141rcu_preempt00:09:173
19820991813,4cyclictest39-21ksoftirqd/322:42:163
1981699180,0cyclictest0-21swapper/223:01:052
1981399189,8cyclictest25-21ksoftirqd/100:11:341
1981399182,4cyclictest9722-21kworker/1:021:10:301
19813991813,1cyclictest0-21swapper/122:31:591
1981399180,2cyclictest0-21swapper/122:23:391
1981399180,2cyclictest0-21swapper/121:05:171
1981399180,1cyclictest0-21swapper/121:37:521
1981399180,18cyclictest0-21swapper/122:56:561
1981399180,16cyclictest0-21swapper/123:35:051
1981099180,2cyclictest0-21swapper/000:35:130
19820991716,1cyclictest0-21swapper/322:16:303
1982099170,0cyclictest0-21swapper/323:29:453
1981699172,2cyclictest0-21swapper/222:09:322
1981699170,2cyclictest0-21swapper/223:45:322
1981699170,1cyclictest0-21swapper/221:56:222
1981699170,1cyclictest0-21swapper/221:17:402
1981699170,0cyclictest0-21swapper/223:14:382
1981399171,1cyclictest0-21swapper/122:08:211
1981399170,1cyclictest0-21swapper/123:59:151
19820991615,1cyclictest0-21swapper/323:37:253
19820991615,1cyclictest0-21swapper/300:38:313
1981699160,0cyclictest0-21swapper/222:35:192
1981399160,1cyclictest0-21swapper/123:18:081
1981399160,1cyclictest0-21swapper/123:10:041
1981099165,6cyclictest20991-21perf20:20:000
19810991615,1cyclictest0-21swapper/000:09:090
19810991610,5cyclictest10025-21apache222:58:220
1981699153,5cyclictest2303-21kworker/u8:000:19:092
19816991512,2cyclictest24059-21cat21:35:262
1981699150,2cyclictest0-21swapper/221:20:242
1981699150,1cyclictest0-21swapper/221:45:182
1981699150,1cyclictest0-21swapper/221:45:172
1981699150,1cyclictest0-21swapper/221:32:212
1981699150,1cyclictest0-21swapper/200:05:142
1981399152,12cyclictest12192-21ssh00:02:381
19813991511,3cyclictest16604-21wc21:50:211
1981399150,14cyclictest0-21swapper/122:25:021
1981399150,13cyclictest5556-21grep20:55:121
1981099152,7cyclictest29455-21ssh23:35:010
1982099142,9cyclictest0-21swapper/322:10:213
19820991413,1cyclictest0-21swapper/300:24:103
19820991412,2cyclictest39-21ksoftirqd/323:40:023
19820991412,2cyclictest39-21ksoftirqd/321:29:313
1982099140,14cyclictest0-21swapper/321:49:283
1982099140,14cyclictest0-21swapper/321:49:273
1982099140,13cyclictest27014-21cat20:30:213
1981699148,3cyclictest0-21swapper/222:45:582
1981699143,6cyclictest0-21swapper/200:10:132
19816991411,2cyclictest7012-21sh00:00:002
1981699140,1cyclictest0-21swapper/222:11:212
1981399142,11cyclictest12397-21diskmemload22:36:441
19813991411,2cyclictest23139-21tail19:15:161
19813991411,2cyclictest13807-21cron20:05:001
19813991410,2cyclictest17125-21tr22:10:151
1981099142,6cyclictest22174-21idleruntime-cro19:15:010
1981099141,7cyclictest5186-21munin-run20:54:590
19810991413,1cyclictest0-21swapper/019:55:120
19810991410,3cyclictest29145-21cron21:20:000
19810991410,3cyclictest18246-21head22:10:220
19810991410,2cyclictest2487-21grep00:15:310
1981099140,1cyclictest0-21swapper/021:55:060
1982099139,3cyclictest6914-21tr19:50:133
1982099139,3cyclictest28430-21tr20:35:143
1982099139,3cyclictest25408-21taskset21:36:143
1982099139,3cyclictest24137-21wc23:50:213
1982099132,7cyclictest12397-21diskmemload21:42:293
1982099131,8cyclictest0-21swapper/322:35:563
19820991312,1cyclictest0-21swapper/322:05:263
19820991310,2cyclictest9908-21tr22:25:153
19820991310,2cyclictest24259-21uniq20:25:153
19820991310,2cyclictest22002-21grep20:20:173
19820991310,2cyclictest19939-21basename23:10:013
19820991310,2cyclictest19939-21basename23:10:003
19820991310,2cyclictest18357-21grep22:30:153
1982099130,13cyclictest0-21swapper/322:48:473
1981699139,2cyclictest23642-21aten2_r7power_c20:25:122
1981699137,5cyclictest10028-21apache222:32:562
19816991310,2cyclictest9777-21cut00:20:182
19816991310,2cyclictest8615-21grep21:00:162
19816991310,2cyclictest29472-21cat22:55:282
19816991310,2cyclictest26094-21tr00:30:152
19816991310,2cyclictest12381-21cron21:10:002
1981699130,3cyclictest0-21swapper/222:28:582
1981699130,1cyclictest0-21swapper/223:54:582
1981699130,1cyclictest0-21swapper/222:53:262
1981699130,13cyclictest0-21swapper/223:37:202
1981399139,3cyclictest12397-21diskmemload23:06:151
1981399139,3cyclictest12397-21diskmemload23:06:151
19813991310,2cyclictest9066-21sed21:00:211
19813991310,2cyclictest4951-21wc23:20:141
19813991310,2cyclictest31998-21grep21:40:181
1981099139,3cyclictest29806-21cut19:30:130
1981099139,2cyclictest14344-21grep21:10:240
1981099138,3cyclictest32408-21cron00:15:000
1981099136,7cyclictest21185-21munin-node22:50:280
1981099136,7cyclictest1035-21packagekitd22:41:130
1981099132,9cyclictest0-21swapper/022:26:230
1981099132,3cyclictest141rcu_preempt21:52:240
19810991310,2cyclictest6171-21ssh23:20:210
19810991310,2cyclictest28499-21tr20:35:150
19810991310,2cyclictest14835-21wc21:50:010
19810991310,2cyclictest14835-21wc21:50:000
19810991310,2cyclictest1482-21grep20:45:170
19810991310,2cyclictest11988-21tr20:00:130
1981099130,7cyclictest16337-21perf22:10:000
1982099129,2cyclictest8206-21grep00:00:223
1982099129,2cyclictest682-21grep22:00:173
1982099129,2cyclictest4862-21tr23:20:133
1982099129,2cyclictest39-21ksoftirqd/323:32:403
1982099129,2cyclictest27237-21date22:55:003
1982099129,2cyclictest25606-21grep19:20:153
1982099129,2cyclictest20579-21tr23:10:143
19820991210,1cyclictest3300-21grep20:50:133
1982099120,12cyclictest0-21swapper/321:17:423
1982099120,11cyclictest32456-21grep20:40:243
1981699129,2cyclictest823-21grep22:00:172
1981699129,2cyclictest30889-21grep21:40:122
1981699129,2cyclictest30787-21grep19:30:212
1981699129,2cyclictest30555-21wc20:39:592
1981699129,2cyclictest2826-21sleep20:50:002
1981699129,2cyclictest17161-21grep20:10:162
1981699129,2cyclictest10525-21ls19:55:222
1981699128,3cyclictest8443-21sh21:26:042
1981699128,3cyclictest17759-21ssh23:27:252
1981699124,2cyclictest4017-21kworker/u8:222:22:202
1981699120,3cyclictest0-21swapper/223:30:442
1981699120,2cyclictest0-21swapper/223:40:432
1981399129,2cyclictest31027-21tr20:40:121
1981399129,2cyclictest29073-21tail20:35:181
1981399129,2cyclictest25712-21cron20:30:001
1981399129,2cyclictest14828-21grep20:05:161
1981399128,3cyclictest2275-21wc20:45:271
1981399128,3cyclictest15220-21cut23:45:181
1981399128,2cyclictest2029-21tr00:15:281
1981399128,2cyclictest17691-21head20:10:221
1981399124,4cyclictest225-21systemd-journal19:25:011
1981099129,2cyclictest9075-21ssh22:25:010
1981099129,2cyclictest7884-21cut21:00:130
1981099129,2cyclictest7115-21tail20:55:240
1981099129,2cyclictest7037-21grep21:25:220
1981099129,2cyclictest524-21wc20:45:000
1981099129,2cyclictest32652-21chrt23:35:560
1981099129,2cyclictest29622-21tail23:15:170
1981099129,2cyclictest25448-21sort19:20:160
1981099129,2cyclictest21847-21grep23:10:220
1981099129,2cyclictest17601-21tr00:25:140
1981099129,2cyclictest13828-21grep23:25:150
1981099128,3cyclictest9702-21head00:20:180
1981099128,3cyclictest32656-21grep19:35:170
1981099128,3cyclictest26105-21head22:15:180
1981099128,3cyclictest11975-21grep21:05:270
1981099128,2cyclictest31076-21tr21:40:130
1981099128,2cyclictest29746-21grep21:20:130
1981099126,5cyclictest14061-21ssh21:30:130
1981099124,7cyclictest18749-21munin-run22:50:000
1982099119,1cyclictest705-21sh00:15:203
1982099118,2cyclictest966-21ssh22:20:043
1982099118,2cyclictest5558-21tr20:55:133
1982099118,2cyclictest32147-21ssh21:20:483
1982099118,2cyclictest32012-21wget19:35:143
1982099118,2cyclictest31009-21taskset21:59:213
1982099118,2cyclictest29822-21tr19:30:143
1982099118,2cyclictest28125-21expr22:55:163
1982099118,2cyclictest26023-21ssh00:30:143
1982099118,2cyclictest2332-21ssh23:56:433
1982099118,2cyclictest19535-21ssh21:33:113
1982099118,2cyclictest18556-21cron20:15:003
1982099118,2cyclictest15458-21grep20:05:223
1982099118,2cyclictest12702-21ssh21:10:033
1982099118,2cyclictest12408-21grep20:00:163
1982099110,5cyclictest0-21swapper/321:50:033
1982099110,10cyclictest27468-21tr19:25:133
1982099110,10cyclictest24526-21taskset19:19:593
1982099110,10cyclictest15750-21awk23:45:223
1981699119,1cyclictest15704-21grep21:50:152
1981699118,2cyclictest8386-21cat19:50:262
1981699118,2cyclictest6461-21taskset19:48:122
1981699118,2cyclictest4208-21cat22:40:242
1981699118,2cyclictest32722-21rm23:55:242
1981699118,2cyclictest32213-21grep19:35:152
1981699118,2cyclictest26419-21cat22:15:212
1981699118,2cyclictest25060-21cut19:20:142
1981699118,2cyclictest21938-21grep20:20:162
1981699118,2cyclictest14820-21grep20:05:162
1981699118,2cyclictest14306-21ls21:10:222
1981699118,2cyclictest11782-21ssh23:24:172
1981699117,3cyclictest394-21ls20:44:592
1981699115,2cyclictest225-21systemd-journal19:29:552
1981699111,6cyclictest0-21swapper/219:45:002
1981699110,9cyclictest0-21swapper/220:30:142
1981399118,2cyclictest5239-21grep19:45:161
1981399118,2cyclictest31746-21wc19:35:011
1981399118,2cyclictest27960-21tail19:25:161
1981399118,2cyclictest22685-21ssh23:50:101
1981399118,2cyclictest1741-21sed19:40:001
1981399111,3cyclictest520-21gmain20:50:111
1981399111,2cyclictest225-21systemd-journal19:45:001
1981099119,1cyclictest26696-21grep22:35:150
1981099119,1cyclictest23443-21ssh21:35:200
1981099118,2cyclictest6950-21ssh23:40:190
1981099118,2cyclictest6152-21grep19:45:250
1981099118,2cyclictest3892-21tr23:00:140
1981099118,2cyclictest31918-21grep23:55:170
1981099118,2cyclictest28101-21ssh00:30:330
1981099118,2cyclictest2734-21ssh22:01:020
1981099118,2cyclictest26920-21uname20:30:190
1981099118,2cyclictest25385-21taskset20:25:520
1981099118,2cyclictest18302-21ssh23:46:570
1981099118,2cyclictest13946-21grep23:05:280
1981099118,2cyclictest13946-21grep23:05:270
1981099117,3cyclictest2902-21wc19:40:180
1981099117,3cyclictest22679-21wc20:20:240
1981099112,8cyclictest18031-21ssh22:30:120
1981099110,10cyclictest28427-21ssh23:53:190
1982099108,1cyclictest9745-21chrt21:02:243
1982099108,1cyclictest9489-21cut19:55:153
1982099108,1cyclictest6156-21grep19:45:243
1982099108,1cyclictest20918-21cat19:10:173
1982099107,2cyclictest2673-21taskset20:48:213
1982099107,2cyclictest2047-21hostname19:40:143
1982099107,2cyclictest10247-21chrt21:05:093
1982099102,1cyclictest0-21swapper/320:15:143
1981699108,1cyclictest5779-21expr20:55:142
1981699108,1cyclictest3426-21expr20:50:132
1981699108,1cyclictest22625-21cat19:15:142
1981699108,1cyclictest21431-21sensors19:10:212
1981699108,1cyclictest11773-21wget20:00:132
1981399108,1cyclictest8006-21ntpq19:50:201
1981399108,1cyclictest21754-21head19:10:251
1981399107,2cyclictest26476-21grep20:30:151
1981399107,2cyclictest10778-21cat19:55:241
1981399100,5cyclictest21432-21tr20:20:141
1981099108,1cyclictest8751-21chrt19:52:090
1981099107,2cyclictest7500-21ssh00:00:190
1981099107,2cyclictest14796-21grep20:05:160
198139991,3cyclictest19098-21cpuspeed_turbos20:15:151
198109997,1cyclictest27686-21sed19:25:140
198109996,2cyclictest17056-21sort20:10:170
198109986,1cyclictest23165-21expr19:15:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional