You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 04:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 14, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
976729668,9sleep30-21swapper/319:05:293
1015229568,8sleep00-21swapper/019:07:220
1017629173,13sleep20-21swapper/219:07:412
1025728768,15sleep10-21swapper/119:08:441
160932670,0sleep30-21swapper/323:13:283
10445992623,2cyclictest1608-21grep19:55:252
1044599260,24cyclictest0-21swapper/222:20:142
10438992523,1cyclictest13-21ksoftirqd/021:53:360
1044399241,14cyclictest0-21swapper/100:10:171
1044899231,21cyclictest0-21swapper/320:25:143
10438992320,2cyclictest13-21ksoftirqd/000:20:000
10438992320,2cyclictest13-21ksoftirqd/000:20:000
1044899220,21cyclictest7072-21kworker/3:120:04:283
1044899212,19cyclictest7072-21kworker/3:120:08:173
1044899210,19cyclictest2017-21taskset19:59:453
1044599210,18cyclictest0-21swapper/222:30:192
1043899217,7cyclictest5372-21perf21:50:010
1043899217,7cyclictest5372-21perf21:50:000
10438992117,3cyclictest13-21ksoftirqd/023:35:400
1044899200,19cyclictest7072-21kworker/3:120:10:133
1044399201,12cyclictest3030-21diskmemload21:51:151
1044599190,19cyclictest0-21swapper/221:23:372
1044399197,5cyclictest141rcu_preempt22:22:041
10443991918,1cyclictest0-21swapper/123:56:591
1044399190,18cyclictest0-21swapper/120:10:181
1043899196,6cyclictest225-21systemd-journal21:22:170
1043899190,1cyclictest0-21swapper/023:55:120
10448991817,1cyclictest7072-21kworker/3:123:27:043
1044899180,18cyclictest15175-21kworker/3:000:34:133
1044899180,16cyclictest4940-21grep21:30:153
1044599186,10cyclictest1-21systemd21:29:562
10445991817,1cyclictest0-21swapper/223:45:582
1044599180,1cyclictest0-21swapper/223:34:472
1044599180,0cyclictest0-21swapper/221:48:042
1044599180,0cyclictest0-21swapper/221:48:032
1044399188,3cyclictest0-21swapper/122:37:481
10443991818,0cyclictest0-21swapper/100:20:461
1043899186,6cyclictest27370-21idleruntime-cro19:45:000
10448991716,1cyclictest7072-21kworker/3:123:09:543
1044899171,15cyclictest0-21swapper/300:35:173
1044599171,15cyclictest0-21swapper/222:50:462
1044599170,17cyclictest0-21swapper/200:19:332
1044599170,17cyclictest0-21swapper/200:19:322
10438991716,1cyclictest0-21swapper/023:34:580
10438991714,1cyclictest13-21ksoftirqd/021:29:550
10438991713,2cyclictest13-21ksoftirqd/022:54:280
10438991711,1cyclictest13-21ksoftirqd/022:18:290
1044599166,10cyclictest16953-21apache222:03:182
10445991615,1cyclictest32-21ksoftirqd/223:35:042
1044399169,4cyclictest22524-21tr00:15:161
1044399169,4cyclictest22524-21tr00:15:151
1043899163,2cyclictest141rcu_preempt19:50:140
1043899163,2cyclictest141rcu_preempt19:50:140
10438991616,0cyclictest0-21swapper/022:56:580
10438991615,1cyclictest13-21ksoftirqd/021:15:590
10438991612,2cyclictest13-21ksoftirqd/023:54:300
1043899160,2cyclictest0-21swapper/021:10:390
1043899160,1cyclictest141rcu_preempt23:02:260
10448991514,1cyclictest7072-21kworker/3:122:15:333
1044899151,13cyclictest0-21swapper/322:14:363
1044599151,2cyclictest141rcu_preempt22:57:512
1044599150,2cyclictest0-21swapper/223:28:552
10443991512,2cyclictest25-21ksoftirqd/121:44:441
1044399150,2cyclictest0-21swapper/122:05:591
1043899154,6cyclictest13-21ksoftirqd/023:10:420
10438991511,2cyclictest13-21ksoftirqd/022:21:070
1043899150,3cyclictest0-21swapper/023:46:410
1043899150,2cyclictest0-21swapper/000:29:230
10445991411,2cyclictest28303-21sleep20:55:002
1044399144,2cyclictest225-21systemd-journal21:29:561
10443991414,0cyclictest0-21swapper/122:30:181
1043899148,4cyclictest13-21ksoftirqd/021:44:420
1043899147,2cyclictest13-21ksoftirqd/023:26:020
1043899142,7cyclictest14830-21munin-run22:15:010
1043899142,7cyclictest13-21ksoftirqd/022:45:160
1043899142,1cyclictest141rcu_preempt19:35:220
1043899141,2cyclictest141rcu_preempt21:55:190
10438991411,2cyclictest27178-21wc20:50:210
10438991411,2cyclictest13-21ksoftirqd/020:20:000
10438991410,3cyclictest30122-21sed23:40:270
1043899140,0cyclictest141rcu_preempt22:33:350
10448991310,2cyclictest26783-21sed23:00:183
10448991310,2cyclictest26472-21grep19:40:233
10448991310,2cyclictest25418-21tr22:59:593
10448991310,2cyclictest24172-21grep00:15:273
10448991310,2cyclictest24172-21grep00:15:263
10448991310,2cyclictest22967-21tr22:00:153
10448991310,2cyclictest17682-21grep22:35:213
1044899130,2cyclictest0-21swapper/323:35:123
1044899130,12cyclictest13922-21grep21:35:173
1044599139,3cyclictest1194-21tr21:05:152
1044599135,7cyclictest699-21apache_processe22:45:132
1044599133,1cyclictest30831-21ps00:00:262
1044599131,11cyclictest22500-21kworker/u8:222:08:552
10445991310,2cyclictest8092-21ls22:10:172
10445991310,2cyclictest32740-21wc22:45:002
10445991310,2cyclictest23846-21ls19:35:212
10445991310,2cyclictest2292-21expr23:05:162
10445991310,2cyclictest22371-21munin-run19:35:012
10445991310,2cyclictest19821-21grep23:15:202
10445991310,2cyclictest15710-21tr19:20:162
1044399139,3cyclictest14533-21cat19:15:241
1044399139,2cyclictest10934-21tr19:10:171
1044399138,4cyclictest9434-21ssh23:47:591
1044399136,6cyclictest3030-21diskmemload00:05:271
10443991312,1cyclictest0-21swapper/100:38:361
10443991310,2cyclictest697-21sed19:55:161
10443991310,2cyclictest5430-21grep20:05:181
10443991310,2cyclictest3007-21cron21:10:001
10443991310,2cyclictest28723-21uname19:45:211
10443991310,2cyclictest20870-21cat23:35:191
10443991310,2cyclictest14884-21grep20:25:171
10443991310,2cyclictest11082-21sed20:15:281
10443991310,2cyclictest0-21swapper/123:20:201
1044399130,2cyclictest0-21swapper/121:16:261
1044399130,13cyclictest0-21swapper/121:11:271
1044399130,12cyclictest3030-21diskmemload21:35:361
1044399130,12cyclictest17057-21wc22:15:251
1043899139,3cyclictest10940-21tr19:10:180
1043899133,1cyclictest141rcu_preempt19:30:170
10438991311,1cyclictest161rcuc/021:34:240
10438991311,1cyclictest13-21ksoftirqd/023:19:480
10438991310,2cyclictest3946-21grep20:00:270
10438991310,2cyclictest31214-21tr21:00:140
10438991310,2cyclictest13802-21cut19:15:160
10438991310,2cyclictest13-21ksoftirqd/023:07:060
10438991310,2cyclictest11964-21grep20:20:140
1044899129,2cyclictest96692sleep323:48:103
1044899129,2cyclictest9340-21grep22:30:253
1044899129,2cyclictest5626-21expr00:05:153
1044899129,2cyclictest32109-21perf19:55:013
1044899129,2cyclictest32109-21perf19:55:003
1044899129,2cyclictest31998-21taskset22:25:103
1044899129,2cyclictest31101-21df00:20:153
1044899129,2cyclictest30551-21grep21:25:263
1044899129,2cyclictest29907-21cat19:50:003
1044899129,2cyclictest27810-21chrt22:41:143
1044899129,2cyclictest24538-21head20:45:183
1044899129,2cyclictest22278-21head23:55:193
1044899129,2cyclictest18364-21head20:30:273
1044899129,2cyclictest15245-21seq23:32:003
1044899129,2cyclictest13886-21grep23:50:203
1044899129,2cyclictest11454-21head19:10:193
1044899128,3cyclictest12616-21grep20:20:173
1044899121,10cyclictest0-21swapper/321:50:113
10448991210,2cyclictest39-21ksoftirqd/322:50:213
1044599129,2cyclictest5810-21grep20:05:212
1044599129,2cyclictest31359-21grep00:20:162
1044599129,2cyclictest2558-21cat23:25:002
1044599129,2cyclictest23559-21grep00:35:172
1044599129,2cyclictest22223-21head20:40:172
1044599129,2cyclictest19811-21cron21:20:012
1044599129,2cyclictest16551-21grep22:15:202
1044599129,2cyclictest15879-21grep20:25:262
1044599129,2cyclictest15065-21cut00:10:252
1044599129,2cyclictest13729-21sh21:55:002
1044599129,2cyclictest11036-21grep23:10:162
10445991212,0cyclictest0-21swapper/200:09:262
10445991211,1cyclictest0-21swapper/220:30:142
10445991210,1cyclictest8313-21sed00:25:252
1044599120,9cyclictest0-21swapper/220:20:132
1044599120,5cyclictest0-21swapper/221:40:592
1044599120,12cyclictest0-21swapper/222:36:212
1044399129,3cyclictest38150irq/131-snd_hda00:03:261
1044399129,2cyclictest7267-21ssh22:10:131
1044399129,2cyclictest6532-21sed21:30:261
1044399129,2cyclictest24061-21grep22:00:221
1044399129,2cyclictest18132-21wc23:15:001
1044399129,2cyclictest13544-21uname23:50:181
1044399129,2cyclictest11568-21grep23:30:151
1044399129,2cyclictest0-21swapper/122:53:351
1044399128,3cyclictest23092-21tail20:40:261
1044399128,2cyclictest3120-21cut20:00:181
1043899129,2cyclictest30833-21wc00:00:260
1043899129,2cyclictest24547-21grep20:45:180
1043899129,2cyclictest23558-21munin-plugin-st20:45:000
1043899129,2cyclictest2042-21cron20:00:010
1043899129,2cyclictest16865-21tr20:30:160
10438991210,1cyclictest13-21ksoftirqd/023:20:530
282652110,1sleep00-21swapper/022:41:360
1044899119,2cyclictest39-21ksoftirqd/322:21:223
1044899118,2cyclictest991-21ssh00:02:063
1044899118,2cyclictest7965-21readlink00:25:213
1044899118,2cyclictest7016-21ssh21:11:363
1044899118,2cyclictest32169-21cat22:05:223
1044899118,2cyclictest30483-21taskset20:57:463
1044899118,2cyclictest30213-21ssh23:21:243
1044899118,2cyclictest29820-21ssh23:40:223
1044899118,2cyclictest2592-21sed21:05:253
1044899118,2cyclictest23644-21ssh21:21:263
1044899118,2cyclictest23222-21ssh21:40:253
1044899118,2cyclictest2243-21ssh21:47:363
1044899118,2cyclictest2243-21ssh21:47:353
1044899118,2cyclictest21798-21cat19:30:273
1044899118,2cyclictest18895-21tr23:15:143
1044899118,2cyclictest1665-21ssh22:45:173
1044899118,2cyclictest15172-21idleruntime21:55:193
1044899118,2cyclictest13921-21ssh00:10:183
1044899118,2cyclictest11945-21ssh21:15:133
1044899118,2cyclictest10378-21awk20:15:213
1044899116,4cyclictest20144-21grep20:35:193
1044599118,2cyclictest5992-21ls21:30:212
1044599118,2cyclictest4818-21ntpq21:10:202
1044599118,2cyclictest32090-21cron19:55:012
1044599118,2cyclictest32090-21cron19:55:002
1044599118,2cyclictest30870-21chrt23:40:572
1044599118,2cyclictest30338-21taskset20:55:542
1044599118,2cyclictest28849-21ssh23:01:102
1044599118,2cyclictest27362-21sleep19:45:002
1044599118,2cyclictest24256-21ssh23:56:282
1044599118,2cyclictest18184-21rm23:53:152
1044599118,2cyclictest14364-21grep21:35:192
1044599118,2cyclictest11606-21tr20:20:012
1044599118,2cyclictest11396-21head19:10:192
1044599117,3cyclictest14111-21ls19:15:182
1044399119,1cyclictest28568-21chrt20:55:131
1044399119,1cyclictest26235-21sed22:40:241
1044399118,2cyclictest4165-21tail23:25:201
1044399118,2cyclictest411-21head22:25:171
1044399118,2cyclictest29850-21ssh23:40:241
1044399118,2cyclictest2602-21grep22:45:261
1044399118,2cyclictest25664-21ssh23:00:021
1044399118,2cyclictest25528-21head20:45:271
1044399118,2cyclictest25508-21ssh21:22:541
1044399118,2cyclictest24401-21taskset19:35:461
1044399118,2cyclictest22402-21ssh00:34:461
1044399118,2cyclictest20924-21grep19:30:181
1044399118,2cyclictest20143-21grep20:35:181
1044399118,2cyclictest19141-21wc19:25:201
1044399118,2cyclictest18893-21grep23:15:141
1044399118,2cyclictest17302-21grep20:30:171
1044399118,2cyclictest16050-21grep19:20:181
1044399118,2cyclictest14371-21cut21:55:151
1044399118,2cyclictest1044-21ssh21:46:391
1044399118,2cyclictest1044-21ssh21:46:391
1044399117,3cyclictest1599-21ssh23:05:031
1044399111,1cyclictest3030-21diskmemload22:55:161
1044399110,10cyclictest0-21swapper/100:26:591
1043899118,2cyclictest5391-21grep00:05:150
1043899118,2cyclictest16079-21ssh22:35:020
1043899118,2cyclictest15014-21cut00:30:140
1043899117,3cyclictest428-21tail22:25:170
1043899112,6cyclictest14018-21hddtemp_smartct21:35:160
1043899110,1cyclictest0-21swapper/022:00:380
1044899108,1cyclictest31052-21cut21:00:143
1044899108,1cyclictest23002-21expr19:35:143
1044899108,1cyclictest21471-21wget20:40:133
1044899108,1cyclictest19261-21grep19:25:213
1044899100,9cyclictest26211-21seq20:50:163
1044899100,10cyclictest17323-21seq19:21:203
1044599108,1cyclictest8911-21chrt20:11:312
1044599108,1cyclictest28209-21taskset19:45:172
1044599108,1cyclictest24589-21head20:45:182
1044599108,1cyclictest19478-21expr20:35:142
1044599108,1cyclictest18662-21tail19:25:162
1044599108,1cyclictest1731-21rm22:25:302
1044599108,1cyclictest16256-21grep00:30:222
1044599107,2cyclictest31082-21basename21:00:132
1044599107,2cyclictest3040-21grep20:00:182
1044599107,2cyclictest13962-21rm21:55:022
1044399107,2cyclictest26944-21grep20:50:201
1044399107,2cyclictest25386-21egrep19:40:151
1044399107,2cyclictest12115-21grep20:20:141
1044399106,2cyclictest30022-21grep19:50:151
1044399106,2cyclictest30022-21grep19:50:141
1043899108,1cyclictest30540-21taskset20:58:310
1043899108,1cyclictest161rcuc/020:10:150
1043899108,1cyclictest14585-21expr20:25:140
1043899108,1cyclictest13816-21ssh00:10:170
1043899108,1cyclictest0-21swapper/000:20:120
1043899107,2cyclictest6220-21wc20:05:270
1043899107,2cyclictest23400-21cut00:35:150
1043899107,2cyclictest19790-21grep20:35:160
1043899107,2cyclictest19667-21chrt19:25:570
1043899100,9cyclictest630-21sh22:05:280
104489997,1cyclictest14965-21taskset19:17:323
104439996,2cyclictest32106-21ntp_states21:00:201
104389997,1cyclictest16341-21cat19:20:190
104389996,2cyclictest29724-21munin-run19:50:010
104389996,2cyclictest1189-21tr21:05:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional