You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 21:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat May 30, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220329280,8sleep00-21swapper/007:07:210
208329165,20sleep20-21swapper/207:05:482
224628366,13sleep10-21swapper/107:07:561
206928269,9sleep30-21swapper/307:05:373
206892740,0sleep20-21swapper/210:22:502
143182730,0sleep2311rcuc/212:34:262
46102430,0sleep00-21swapper/011:30:200
2495992828,0cyclictest0-21swapper/209:43:042
249599230,1cyclictest0-21swapper/212:15:132
249599210,21cyclictest0-21swapper/211:01:562
2490992120,1cyclictest0-21swapper/108:06:341
249099210,0cyclictest0-21swapper/110:15:201
249599200,19cyclictest0-21swapper/210:15:122
249099200,18cyclictest0-21swapper/112:22:331
2497991919,0cyclictest0-21swapper/312:12:263
249599190,1cyclictest0-21swapper/211:22:502
249099190,17cyclictest0-21swapper/108:39:591
249099183,1cyclictest141rcu_preempt10:25:131
2490991813,4cyclictest25-21ksoftirqd/111:20:011
249099180,18cyclictest0-21swapper/111:42:071
248699185,7cyclictest16228-21munin-run10:40:010
249599170,15cyclictest0-21swapper/212:10:442
2490991716,1cyclictest0-21swapper/109:42:131
249099170,8cyclictest0-21swapper/110:08:131
249099170,17cyclictest0-21swapper/110:41:571
249099170,15cyclictest30495-21ssh09:30:211
249099170,15cyclictest30495-21ssh09:30:211
249099170,0cyclictest0-21swapper/110:02:481
2486991717,0cyclictest0-21swapper/012:33:560
2497991613,1cyclictest0-21swapper/311:35:193
2497991612,2cyclictest14485-21apt-get12:15:133
249799160,15cyclictest0-21swapper/310:15:183
249799160,0cyclictest0-21swapper/311:18:523
249599160,16cyclictest0-21swapper/209:31:012
249599160,16cyclictest0-21swapper/209:31:002
249099161,1cyclictest141rcu_preempt11:16:511
249099160,1cyclictest0-21swapper/110:50:011
249099160,16cyclictest0-21swapper/111:02:351
248699160,1cyclictest0-21swapper/010:42:470
167872160,0sleep20-21swapper/211:18:402
249799150,15cyclictest0-21swapper/309:20:023
249599150,15cyclictest0-21swapper/209:48:032
249599150,0cyclictest0-21swapper/211:50:052
2490991513,0cyclictest0-21swapper/111:31:591
2490991512,2cyclictest12977-21sleep09:40:011
248699151,8cyclictest9835-21munin-run07:25:010
2486991513,2cyclictest13-21ksoftirqd/011:59:190
249799148,5cyclictest39-21ksoftirqd/311:26:093
2495991414,0cyclictest0-21swapper/210:51:072
2495991413,1cyclictest55650irq/127-enp2s0-10:04:582
2495991412,1cyclictest0-21swapper/209:05:222
249599140,12cyclictest0-21swapper/211:37:352
2490991414,0cyclictest0-21swapper/112:01:551
2490991411,2cyclictest25-21ksoftirqd/111:25:111
2490991410,3cyclictest32565-21ssh11:46:411
2490991410,2cyclictest8030-21grep12:10:251
2490991410,2cyclictest25-21ksoftirqd/109:20:011
249099140,14cyclictest0-21swapper/110:38:431
249099140,13cyclictest0-21swapper/111:51:351
249099140,12cyclictest16556-21grep07:35:261
249099140,12cyclictest16556-21grep07:35:251
248699141,7cyclictest3388-21munin-run09:14:590
2486991410,2cyclictest5860-21grep09:15:280
195802140,0chrt0-21swapper/009:24:250
2497991310,2cyclictest18179-21cut10:40:203
2497991310,2cyclictest16482-21grep10:20:203
2497991310,2cyclictest13746-21awk07:30:213
2497991310,2cyclictest10766-21sh08:34:593
249799130,1cyclictest0-21swapper/310:50:093
249599139,3cyclictest19023-21cut08:50:182
249599139,3cyclictest10760-21cron08:34:592
249599139,2cyclictest10772-21cat10:55:212
249599134,8cyclictest25812-21kworker/u8:112:25:442
249599132,7cyclictest477-21rs:main2
249599131,2cyclictest891-21ssh12:06:222
2495991310,2cyclictest5536-21grep08:20:252
2495991310,2cyclictest26038-21sed07:55:272
2495991310,2cyclictest24988-21sort10:45:012
2495991310,2cyclictest23554-21grep09:00:152
2495991310,2cyclictest21258-21grep08:55:152
2495991310,2cyclictest15646-21sed07:35:162
2495991310,2cyclictest15646-21sed07:35:152
249599130,13cyclictest0-21swapper/211:41:042
249099139,2cyclictest13779-21wc07:30:221
249099138,4cyclictest0-21swapper/112:35:251
249099133,4cyclictest225-21systemd-journal09:29:331
249099131,10cyclictest0-21swapper/112:15:131
2490991310,2cyclictest6395-21tr09:55:151
2490991310,2cyclictest31482-21chrt12:25:141
2490991310,2cyclictest29646-21basename09:50:011
2490991310,2cyclictest23678-21grep07:50:281
2490991310,2cyclictest23258-21tr09:00:141
2490991310,2cyclictest19393-21cron07:45:011
248699139,3cyclictest6933-21grep07:15:270
248699139,3cyclictest30140-21ssh10:09:310
248699139,3cyclictest19417-21awk07:45:000
248699139,2cyclictest13658-21tr07:30:210
248699135,7cyclictest1370-21apache208:20:120
248699135,2cyclictest10116-21idleruntime-cro11:15:000
248699131,6cyclictest22350-21munin-run12:20:000
2486991310,2cyclictest23456-21cron12:40:010
2486991310,2cyclictest1889-21cat11:10:000
248699130,9cyclictest0-21swapper/012:23:330
248699130,13cyclictest0-21swapper/010:58:150
248699130,12cyclictest30436-21uname09:30:210
248699130,12cyclictest30436-21uname09:30:200
292062120,0sleep30-21swapper/309:30:143
292062120,0sleep30-21swapper/309:30:133
249799129,2cyclictest738-21sed10:30:153
249799129,2cyclictest6701-21apt10:14:593
249799129,2cyclictest30789-21grep11:45:233
249799129,2cyclictest30235-21wget09:50:133
249799129,2cyclictest28014-21ssh09:10:143
249799129,2cyclictest26541-21uname10:45:193
249799129,2cyclictest20657-21grep11:40:143
249799129,2cyclictest20417-21grep07:45:163
249799129,2cyclictest196572sleep308:50:273
249799129,2cyclictest1788-21grep08:15:143
249799129,2cyclictest17540-21grep07:40:143
249799129,2cyclictest16177-21wc10:00:243
249799129,2cyclictest13738-21grep08:40:133
249799129,2cyclictest10907-21ssh10:55:223
2497991212,0cyclictest0-21swapper/311:09:433
249599129,2cyclictest9814-21cron07:24:592
249599129,2cyclictest6800-21sort09:55:172
249599129,2cyclictest29917-21grep08:05:192
249599129,2cyclictest28449-21sed08:00:272
249599129,2cyclictest26572-21cut11:05:162
249599129,2cyclictest25717-21taskset12:21:072
249599129,2cyclictest24146-21grep10:05:212
249599129,2cyclictest21728-21cut12:00:122
249599129,2cyclictest19896-21grep07:45:142
249599129,2cyclictest1629-21wc10:30:232
249599129,2cyclictest12452-21mailstats08:35:232
249599129,2cyclictest10635-21grep07:25:162
249599128,3cyclictest7419-21cron07:20:002
249599127,2cyclictest19399-21idleruntime-cro07:45:012
249099129,2cyclictest9424-21sed08:30:171
249099129,2cyclictest5578-21grep08:20:261
249099129,2cyclictest2832-21cut07:10:151
249099129,2cyclictest14592-21head08:40:191
249099129,2cyclictest1428-21cut10:50:151
249099128,3cyclictest17621-21tail10:20:291
249099128,2cyclictest2577-21tr11:10:131
2490991212,0cyclictest0-21swapper/110:31:351
249099120,9cyclictest0-21swapper/109:20:151
249099120,12cyclictest0-21swapper/111:08:551
249099120,11cyclictest290762sleep108:05:001
249099120,10cyclictest0-21swapper/109:50:051
248699129,2cyclictest6798-21grep09:55:170
248699129,2cyclictest3498-21head07:10:190
248699129,2cyclictest32616-21grep12:25:220
248699129,2cyclictest32146-21head10:10:180
248699129,2cyclictest32023-21grep09:50:260
248699129,2cyclictest30916-21tr12:05:170
248699129,2cyclictest28913-21cron08:05:000
248699129,2cyclictest22663-21grep07:50:180
248699129,2cyclictest1857-21grep10:30:240
248699129,2cyclictest17260-21ssh10:20:260
2486991210,1cyclictest2477-21cyclictest11:01:000
2486991210,1cyclictest11510-21apache209:41:480
248699120,1cyclictest141rcu_preempt10:03:290
13632120,2sleep10-21swapper/110:10:521
249799118,2cyclictest9416-21sed07:20:263
249799118,2cyclictest9334-21ssh09:18:083
249799118,2cyclictest8082-21sh09:36:143
249799118,2cyclictest7855-21cat08:25:263
249799118,2cyclictest5548-21cut07:15:153
249799118,2cyclictest5205-21grep11:50:163
249799118,2cyclictest4149-21grep11:30:163
249799118,2cyclictest3732-21grep11:10:213
249799118,2cyclictest31903-21cat12:05:253
249799118,2cyclictest31269-21hostname12:25:133
249799118,2cyclictest27617-21ssh12:22:343
249799118,2cyclictest27547-21date09:10:003
249799118,2cyclictest2687-21sshd07:10:063
249799118,2cyclictest25093-21taskset09:04:473
249799118,2cyclictest24101-21cat10:05:213
249799118,2cyclictest23454-21ssh11:03:063
249799118,2cyclictest22967-21ssh09:25:333
249799118,2cyclictest22667-21head09:45:183
249799118,2cyclictest20630-21ssh11:20:233
249799118,2cyclictest17360-21ssh11:57:033
249799118,2cyclictest16215-21expr08:45:153
249799118,2cyclictest14479-21ssh12:34:343
249799118,2cyclictest13792-21rm10:38:073
249799118,2cyclictest12204-21chrt07:29:333
249799118,2cyclictest11589-21ssh09:58:093
249799116,4cyclictest9597-21kworker/u8:109:41:113
249799116,2cyclictest225-21systemd-journal08:05:013
249799110,10cyclictest24719-21cat07:55:163
249799110,10cyclictest16366-21grep12:35:153
249599118,2cyclictest8049-21ssh10:35:002
249599118,2cyclictest5718-21ssh11:11:302
249599118,2cyclictest5472-21sendmail_mailtr09:15:242
249599118,2cyclictest4811-21ssh09:35:022
249599118,2cyclictest3988-21sh11:30:162
249599118,2cyclictest32708-21wc10:10:232
249599118,2cyclictest31828-21ssh11:46:062
249599118,2cyclictest27979-21chrt10:45:582
249599118,2cyclictest27727-21kworker/u8:109:11:162
249599118,2cyclictest20171-21ssh09:24:532
249599118,2cyclictest14834-21cut08:40:222
249599118,2cyclictest1410-21ssh11:28:542
249599117,3cyclictest32-21ksoftirqd/210:25:112
249599114,5cyclictest31021-21hddtemp_smartct09:50:172
249599112,8cyclictest16731-21/usr/sbin/munin12:35:182
249099119,2cyclictest25-21ksoftirqd/108:10:251
249099118,2cyclictest28832-21grep09:10:181
249099118,2cyclictest26958-21wc09:05:271
249099118,2cyclictest20750-21grep07:45:201
249099118,2cyclictest16896-21sort08:45:181
249099118,2cyclictest16786-21ssh11:56:361
249099118,2cyclictest1348-21ssh12:06:431
249099118,2cyclictest12400-21sh10:56:011
249099118,2cyclictest11968-21chrt07:26:301
249099118,2cyclictest10475-21ssh12:31:291
249099116,2cyclictest14616-21sh11:35:531
249099110,11cyclictest0-21swapper/108:50:141
248699118,2cyclictest7806-21ssh09:36:020
248699118,2cyclictest7071-21ssh10:53:250
248699118,2cyclictest6600-21ssh12:10:150
248699118,2cyclictest5191-21expr11:50:150
248699118,2cyclictest3646-21perf08:20:000
248699118,2cyclictest32273-21head08:10:170
248699118,2cyclictest29971-21uname11:45:170
248699118,2cyclictest29787-21sort08:05:180
248699118,2cyclictest29426-21ssh10:28:060
248699118,2cyclictest28241-21tail11:25:150
248699118,2cyclictest27747-21ssh10:45:480
248699118,2cyclictest24787-21ssh12:01:180
248699118,2cyclictest23369-21sendmail_mailtr09:45:230
248699118,2cyclictest23333-21ssh11:41:090
248699118,2cyclictest20051-21chrt08:51:200
248699118,2cyclictest19983-21cat11:20:180
248699118,2cyclictest18099-21cat08:49:590
248699118,2cyclictest14375-21ssh11:35:420
248699118,2cyclictest11880-21ssh10:17:280
248699117,3cyclictest11849-21cat11:15:200
248699117,2cyclictest20883-21ssh09:25:140
2486991110,1cyclictest0-21swapper/007:55:150
249799108,2cyclictest12216-21ntpq08:35:203
249799108,1cyclictest9762sleep308:11:473
249799108,1cyclictest21998-21taskset07:50:073
249799108,1cyclictest16712-21taskset07:36:253
249799108,1cyclictest16712-21taskset07:36:243
249799107,2cyclictest4161-21grep08:20:143
249799107,2cyclictest23857-21ssh10:25:063
249799107,2cyclictest21740-21ssh12:00:133
249799100,9cyclictest29436-21cut08:05:173
249599108,1cyclictest2369-21if_enp2s008:15:182
249599108,1cyclictest23395-21which07:50:252
249599107,2cyclictest7047-21head08:25:172
249599107,2cyclictest2926-21sort07:10:152
249599107,2cyclictest14110-21head11:55:162
249599100,10cyclictest16874-21load08:45:182
249099108,1cyclictest6378-21grep08:25:141
249099108,1cyclictest26222-21taskset07:56:171
249099108,1cyclictest223562sleep108:55:291
249099107,2cyclictest9219-21wc07:20:251
249099107,2cyclictest2739-21head08:15:211
248699108,1cyclictest20814-21grep08:55:140
248699108,1cyclictest16091-21cut07:35:210
248699108,1cyclictest16091-21cut07:35:200
248699108,1cyclictest10393-21cat07:25:140
248699107,2cyclictest6248-21chrt08:25:020
248699107,2cyclictest19854-21ls07:45:140
248699107,2cyclictest13675-21grep08:40:140
248699107,2cyclictest11257-21grep08:35:140
248699107,2cyclictest10363-21sed08:30:260
24979997,1cyclictest21204-21forks08:55:153
24959998,1cyclictest13173-21grep07:30:162
24959997,1cyclictest32710-21taskset08:10:202
24909997,1cyclictest6402-21needreboot07:15:201
24869997,1cyclictest25354-21taskset09:05:100
24869997,1cyclictest24799-21taskset09:01:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional