You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 18:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Apr 21, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3071129885,9sleep00-21swapper/007:06:220
3073329268,8sleep20-21swapper/207:06:402
2886028871,13sleep10-21swapper/107:05:051
3096328459,9sleep30-21swapper/307:09:383
31088993119,11cyclictest39-21ksoftirqd/312:25:023
3108499270,26cyclictest0-21swapper/209:24:322
3108899250,25cyclictest0-21swapper/311:53:563
3108499250,0cyclictest0-21swapper/210:57:592
31081992524,1cyclictest0-21swapper/110:53:001
31088992319,4cyclictest39-21ksoftirqd/310:21:433
3108899230,23cyclictest0-21swapper/310:53:363
31088992220,1cyclictest23912-21diskmemload11:45:003
3108499220,1cyclictest0-21swapper/210:33:092
3108899212,18cyclictest0-21swapper/312:10:113
31088992118,2cyclictest21834-21ssh11:05:143
31088992115,2cyclictest39-21ksoftirqd/309:56:243
3108499210,21cyclictest0-21swapper/210:25:402
3108499210,0cyclictest0-21swapper/212:05:212
3108199212,1cyclictest22694-21hddtemp_smartct11:05:181
31088992012,6cyclictest39-21ksoftirqd/309:27:273
3108899201,16cyclictest39-21ksoftirqd/311:31:533
3108899201,16cyclictest39-21ksoftirqd/311:31:533
31084992019,1cyclictest0-21swapper/207:57:572
3108499200,19cyclictest0-21swapper/209:33:272
3108499200,15cyclictest0-21swapper/209:35:132
31081992020,0cyclictest0-21swapper/112:30:221
3108199200,20cyclictest0-21swapper/111:04:421
31079992019,1cyclictest0-21swapper/009:34:270
3108899193,15cyclictest0-21swapper/310:39:243
3108899190,0cyclictest0-21swapper/309:21:463
3107999190,1cyclictest0-21swapper/011:35:400
31088991816,1cyclictest18412-21grep12:00:263
31088991816,1cyclictest126122sleep312:16:383
31088991815,2cyclictest23912-21diskmemload10:42:413
3108499180,14cyclictest0-21swapper/212:37:572
3108499180,10cyclictest0-21swapper/210:40:152
31081991815,2cyclictest27082-21tail12:25:191
3107999182,15cyclictest0-21swapper/011:12:070
31079991810,7cyclictest21514-21apache212:29:160
31088991712,5cyclictest39-21ksoftirqd/311:20:353
31088991711,6cyclictest39-21ksoftirqd/310:08:273
31088991711,5cyclictest10415-21/usr/sbin/munin12:35:273
3108899170,2cyclictest0-21swapper/311:25:143
3108899170,2cyclictest0-21swapper/310:56:503
3108899170,1cyclictest0-21swapper/312:21:323
3108499170,1cyclictest0-21swapper/211:00:262
3108499170,16cyclictest0-21swapper/210:00:232
31081991714,1cyclictest0-21swapper/110:35:251
3108199170,14cyclictest0-21swapper/109:53:081
3107999173,1cyclictest141rcu_preempt12:13:110
31079991712,3cyclictest19629-21ls07:50:260
31079991712,3cyclictest19629-21ls07:50:260
3107999170,2cyclictest0-21swapper/009:40:040
3107999170,2cyclictest0-21swapper/009:40:040
3108899168,5cyclictest22983-21grep08:00:013
3108899167,9cyclictest23912-21diskmemload11:03:573
3108899163,4cyclictest39-21ksoftirqd/310:30:053
31088991615,1cyclictest39-21ksoftirqd/309:41:323
31088991615,1cyclictest39-21ksoftirqd/309:41:323
31088991615,1cyclictest381rcuc/311:19:183
31088991614,1cyclictest381rcuc/309:19:443
3108899161,13cyclictest39-21ksoftirqd/310:00:293
3108499160,2cyclictest0-21swapper/210:06:022
3108499160,16cyclictest0-21swapper/210:50:102
31081991614,1cyclictest25-21ksoftirqd/109:13:361
3108199160,16cyclictest0-21swapper/111:21:501
3108199160,16cyclictest0-21swapper/111:17:081
3108199160,16cyclictest0-21swapper/109:32:061
3108199160,16cyclictest0-21swapper/109:18:341
31079991615,1cyclictest0-21swapper/011:54:380
3107999160,1cyclictest0-21swapper/009:28:320
31088991514,1cyclictest39-21ksoftirqd/309:31:413
31088991511,3cyclictest3668-21awk12:30:223
3108899150,2cyclictest0-21swapper/310:19:063
3108499150,5cyclictest0-21swapper/211:19:322
3108499150,5cyclictest0-21swapper/210:22:492
3108499150,2cyclictest0-21swapper/211:30:352
3108499150,2cyclictest0-21swapper/211:30:342
3108499150,1cyclictest0-21swapper/210:48:282
3108499150,14cyclictest0-21swapper/212:25:192
3108499150,14cyclictest0-21swapper/209:00:122
31081991513,1cyclictest38150irq/131-snd_hda09:39:561
31081991512,3cyclictest38150irq/131-snd_hda10:01:121
3108199150,1cyclictest0-21swapper/110:55:371
3108199150,15cyclictest0-21swapper/109:25:351
3107999157,7cyclictest9941-21awk12:35:000
3107999152,3cyclictest225-21systemd-journal12:18:150
31079991511,2cyclictest3404-21perf07:20:000
3107999150,2cyclictest0-21swapper/010:26:390
3107999150,2cyclictest0-21swapper/010:04:390
3107999150,1cyclictest0-21swapper/010:32:480
268262150,1sleep20-21swapper/209:10:462
3108899148,5cyclictest39-21ksoftirqd/311:39:383
3108899147,6cyclictest0-21swapper/311:55:243
31088991412,2cyclictest39-21ksoftirqd/312:08:233
31088991412,2cyclictest39-21ksoftirqd/310:27:233
31088991412,1cyclictest0-21swapper/310:45:123
31088991411,3cyclictest39-21ksoftirqd/311:13:173
31088991411,3cyclictest39-21ksoftirqd/310:10:213
31088991411,2cyclictest24424-21sed08:00:223
31084991410,3cyclictest9900-21basename08:40:142
31084991410,2cyclictest4503-21ntpq10:15:202
3108499140,1cyclictest0-21swapper/211:25:562
3108499140,13cyclictest0-21swapper/212:11:582
3108499140,0cyclictest0-21swapper/211:08:042
3108199145,8cyclictest21510-21apache212:23:161
31081991413,1cyclictest0-21swapper/110:41:511
31081991411,2cyclictest18281-21sh12:40:001
3108199141,11cyclictest0-21swapper/111:30:131
3108199141,11cyclictest0-21swapper/111:30:131
3108199140,2cyclictest25322-21taskset11:45:221
3108199140,14cyclictest0-21swapper/111:13:081
3107999141,7cyclictest11982-21perf08:45:000
31079991414,0cyclictest0-21swapper/011:15:190
31079991414,0cyclictest0-21swapper/010:56:090
31079991413,1cyclictest0-21swapper/010:09:070
31079991411,2cyclictest23376-21expr11:25:140
31079991410,3cyclictest31710-21cron11:50:000
3107999140,1cyclictest0-21swapper/007:10:130
3107999140,14cyclictest0-21swapper/009:14:020
3107999140,13cyclictest29675-21grep08:10:260
3108899131,12cyclictest39-21ksoftirqd/309:46:103
31088991310,3cyclictest0-21swapper/309:37:553
31088991310,2cyclictest13432-21grep07:40:143
31088991310,2cyclictest11060-21tr07:35:143
3108499139,3cyclictest19463-21fgrep07:50:222
3108499139,3cyclictest19463-21fgrep07:50:212
3108499133,7cyclictest17096-21grep12:00:152
31084991310,2cyclictest4891-21ssh10:35:182
31084991310,2cyclictest3512-21grep07:20:002
31084991310,2cyclictest17393-21grep07:45:272
31084991310,2cyclictest14633-21grep07:40:222
3108499130,1cyclictest0-21swapper/212:30:352
3108499130,12cyclictest0-21swapper/209:26:132
3108499130,0cyclictest0-21swapper/212:24:192
3108199139,3cyclictest26050-21cat09:50:011
31081991312,1cyclictest38150irq/131-snd_hda10:23:561
31081991310,2cyclictest7338-21cat08:35:001
31081991310,2cyclictest4316-21grep07:20:161
31081991310,2cyclictest32074-21grep07:10:171
31081991310,2cyclictest1524-21grep07:15:151
3108199130,2cyclictest0-21swapper/110:32:131
3108199130,11cyclictest0-21swapper/110:47:421
3107999131,7cyclictest2740-21munin-run10:15:010
3107999131,6cyclictest22947-21munin-run08:00:010
31079991310,2cyclictest24958-21grep12:05:140
3107999130,12cyclictest24178-21cut08:00:180
192042130,0sleep00-21swapper/010:44:080
3108899128,3cyclictest4403-21head07:20:173
3108899126,5cyclictest0-21swapper/311:45:293
3108899120,12cyclictest0-21swapper/309:54:233
3108499129,2cyclictest6211-21tr08:30:222
3108499129,2cyclictest28712-21head08:10:152
3108499129,2cyclictest25331-21cron08:05:012
3108499129,2cyclictest10929-21tr07:35:142
3108499128,3cyclictest4824-21munin-run08:30:012
3108499128,2cyclictest410-21ssh09:15:142
3108499120,9cyclictest0-21swapper/209:52:162
3108499120,3cyclictest0-21swapper/210:10:122
3108499120,11cyclictest26257-21sort08:05:162
3108199129,2cyclictest9532-21awk11:55:261
3108199129,2cyclictest9476-21chrt08:39:021
3108199129,2cyclictest8166-21ssh11:35:191
3108199129,2cyclictest28798-21ls10:10:211
3108199129,2cyclictest24244-21grep11:25:191
3108199129,2cyclictest22188-21grep07:55:261
3108199129,2cyclictest1563-21cat11:50:271
3108199129,2cyclictest14498-21grep07:40:211
3108199129,2cyclictest12411-21grep07:35:261
3108199128,3cyclictest4024-21grep08:25:231
3108199128,3cyclictest2449-21ls08:25:011
3108199124,4cyclictest273-21systemd-udevd11:42:041
3108199123,8cyclictest3227-21kworker/u8:212:12:461
3107999129,2cyclictest946-21cut08:20:150
3107999129,2cyclictest9361-21ssh11:55:250
3107999129,2cyclictest9143-21basename09:40:000
3107999129,2cyclictest8688-21cut07:30:130
3107999129,2cyclictest31405-21grep11:30:140
3107999129,2cyclictest31405-21grep11:30:140
3107999129,2cyclictest26382-21grep08:05:170
3107999129,2cyclictest22832-21wc09:05:190
3107999129,2cyclictest17614-21wc12:00:190
3107999129,2cyclictest16211-21grep11:40:160
3107999128,3cyclictest5315-21tail07:20:260
3107999128,3cyclictest13535-21head10:20:270
3107999120,2cyclictest0-21swapper/007:25:130
3108899119,1cyclictest0-21swapper/307:15:213
3108899118,2cyclictest4628-21taskset08:27:423
3108899118,2cyclictest39-21ksoftirqd/308:20:013
3108899118,2cyclictest28478-21ssh09:12:033
3108899118,2cyclictest18585-21grep08:55:253
3108899118,2cyclictest16496-21chrt08:52:273
3108899118,2cyclictest13490-21cut08:45:213
3108899118,2cyclictest1053-21head08:20:153
3108899117,3cyclictest11084-21grep08:40:223
3108899114,6cyclictest0-21swapper/307:30:213
3108899113,5cyclictest39-21ksoftirqd/308:10:013
3108499118,2cyclictest9137-21head07:30:172
3108499118,2cyclictest7405-21cut11:35:152
3108499118,2cyclictest3938-21tr07:20:142
3108499118,2cyclictest3563-21ssh11:13:172
3108499118,2cyclictest32090-21ssh11:50:082
3108499118,2cyclictest31693-21cat08:15:222
3108499118,2cyclictest29338-21ssh11:48:092
3108499118,2cyclictest217992sleep209:05:122
3108499118,2cyclictest19094-21munin-plugin-st08:59:592
3108499118,2cyclictest18039-21rm09:45:102
3108499118,2cyclictest17614-21ssh11:21:132
3108499118,2cyclictest15274-21ssh11:40:022
3108499118,2cyclictest15087-21df08:50:152
3108499118,2cyclictest11032-21uname09:40:212
3108499118,2cyclictest11032-21uname09:40:202
3108499118,2cyclictest10630-21ssh11:55:512
3108499117,3cyclictest9564-21sh08:40:002
3108499117,3cyclictest31591-21grep07:10:142
3108499117,3cyclictest144482sleep212:18:072
3108499111,9cyclictest0-21swapper/209:57:372
3108199118,2cyclictest3940-21cut10:15:161
3108199118,2cyclictest3666-21chrt09:55:201
3108199118,2cyclictest25590-21cat08:05:111
3108199118,2cyclictest23451-21sh10:07:161
3108199118,2cyclictest21146-21nfs4_client10:25:201
3108199118,2cyclictest19858-21grep09:00:161
3108199118,2cyclictest19069-21cron09:00:001
3108199118,2cyclictest18610-21ssh12:00:331
3108199118,2cyclictest16065-21ssh09:43:571
3108199118,2cyclictest16065-21ssh09:43:561
3108199118,2cyclictest10614-21ssh12:15:221
3108199117,3cyclictest17005-21tr07:45:221
3108199117,2cyclictest28254-21grep08:10:141
3107999118,2cyclictest8273-21chrt09:58:280
3107999118,2cyclictest6869-21rm10:35:590
3107999118,2cyclictest3964-21tune2fs10:15:160
3107999118,2cyclictest32543-21ssh09:15:030
3107999118,2cyclictest32287-21ssh10:51:170
3107999118,2cyclictest28543-21sh10:49:540
3107999118,2cyclictest26348-21ssh09:50:040
3107999118,2cyclictest23542-21grep11:05:260
3107999118,2cyclictest21838-21ssh12:22:090
3107999118,2cyclictest20015-21sh11:23:070
3107999118,2cyclictest16864-21ssh11:01:350
3107999118,2cyclictest16180-21ssh12:38:220
3107999118,2cyclictest11149-21ssh09:20:460
3107999114,4cyclictest18302-21aten2_r7power_v09:45:140
3107999110,10cyclictest21596-21date09:05:000
3108899109,1cyclictest31443-21cut07:10:143
3108899108,1cyclictest7812-21taskset07:25:493
3108899108,1cyclictest7517-21cut08:35:153
3108899108,1cyclictest30270-21wc08:15:003
3108899108,1cyclictest23881-21chrt09:09:493
3108899108,1cyclictest193612sleep309:00:123
3108899108,1cyclictest16118-21expr07:45:153
3108899107,2cyclictest5061-21basename08:30:133
3108899107,2cyclictest19240-21grep07:50:203
3108899107,2cyclictest19240-21grep07:50:193
3108499108,1cyclictest13659-21cat08:45:232
3108499107,2cyclictest1088-21cut08:20:162
3108499100,9cyclictest6578-21sed07:25:152
3108199108,1cyclictest9824-21cat08:40:121
3108199108,1cyclictest8893-21df07:30:141
3108199108,1cyclictest24827-21apache_processe12:05:111
3108199108,1cyclictest23559-21expr08:00:141
3108199108,1cyclictest18110-21chrt07:50:101
3108199108,1cyclictest18110-21chrt07:50:101
3108199108,1cyclictest163952sleep108:51:091
3108199108,1cyclictest13933-21sh09:22:591
3108199107,2cyclictest32642-21grep08:20:011
3108199107,2cyclictest22941-21tr09:05:201
3108199105,2cyclictest225-21systemd-journal07:26:041
3107999108,1cyclictest7016-21taskset08:32:540
3107999108,1cyclictest4098-21cut08:25:240
3107999108,1cyclictest11232-21cat07:35:150
3107999107,2cyclictest8228-21grep08:35:170
3107999107,2cyclictest32403-21taskset08:19:170
3107999107,2cyclictest17089-21cut08:55:140
3107999107,2cyclictest13824-21ls07:40:160
3107999106,3cyclictest17883-21ls07:50:000
3107999106,3cyclictest15830-21tr08:50:210
310819997,1cyclictest14197-21chrt08:48:231
310799997,1cyclictest12682-21expr08:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional