You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 16:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Sep 12, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2444329173,13sleep00-21swapper/007:09:350
2436729178,9sleep20-21swapper/207:08:362
2417928673,9sleep30-21swapper/307:06:133
2414228265,13sleep10-21swapper/107:05:421
224672580,0sleep00-21swapper/011:09:200
2456799250,24cyclictest0-21swapper/212:35:092
2456799250,24cyclictest0-21swapper/212:33:522
2456799240,23cyclictest0-21swapper/212:08:372
2456999230,23cyclictest0-21swapper/310:48:573
24561992220,2cyclictest13-21ksoftirqd/009:42:290
2456799210,20cyclictest0-21swapper/211:23:402
2456499210,2cyclictest0-21swapper/111:42:151
2456199210,2cyclictest0-21swapper/009:46:400
2456999200,19cyclictest0-21swapper/312:19:023
2456799200,1cyclictest0-21swapper/211:13:492
2456799200,1cyclictest0-21swapper/211:13:482
80552190,1sleep08054-21ssh10:02:050
2456799194,13cyclictest0-21swapper/211:47:102
24564991813,5cyclictest0-21swapper/110:58:231
2456999174,2cyclictest17346-21diskmemload09:34:403
2456999170,2cyclictest0-21swapper/311:52:493
24567991717,0cyclictest0-21swapper/212:18:452
24567991715,2cyclictest55550irq/126-enp2s0-09:25:122
2456499170,17cyclictest0-21swapper/110:25:141
2456199177,3cyclictest22613-21/usr/sbin/munin10:50:100
24561991714,2cyclictest13-21ksoftirqd/011:45:220
2456199170,9cyclictest537-21snmpd08:40:120
2456199170,17cyclictest0-21swapper/010:37:040
2456999162,9cyclictest22696-21ssh10:50:113
24569991616,0cyclictest0-21swapper/310:22:113
24567991614,1cyclictest0-21swapper/210:40:162
2456799160,15cyclictest0-21swapper/210:09:202
24564991611,4cyclictest0-21swapper/109:23:041
2456499160,1cyclictest0-21swapper/112:25:161
2456499160,15cyclictest0-21swapper/107:45:261
2456199166,2cyclictest0-21swapper/012:32:480
2456199160,2cyclictest141rcu_preempt11:43:150
2456199160,15cyclictest0-21swapper/010:08:490
2456799150,15cyclictest0-21swapper/211:38:532
2456799150,15cyclictest0-21swapper/209:15:332
2456799150,14cyclictest0-21swapper/210:57:232
24564991512,2cyclictest4434-21sed10:19:591
2456499150,2cyclictest0-21swapper/111:09:121
2456499150,15cyclictest0-21swapper/112:31:441
2456199154,6cyclictest9951-21perf09:25:000
24561991511,3cyclictest10331-21wc07:45:190
2456199151,12cyclictest6786-21basename07:40:140
2456999142,4cyclictest0-21swapper/311:20:183
24569991412,1cyclictest39-21ksoftirqd/312:30:043
2456999141,1cyclictest0-21swapper/311:11:513
2456999141,1cyclictest0-21swapper/311:11:503
24569991410,3cyclictest12324-21sed09:25:243
2456799148,4cyclictest2483-21ssh11:16:032
24567991410,3cyclictest9790-21sort11:40:192
24567991410,3cyclictest11156-21cut12:20:132
2456799140,13cyclictest2370-21ssh09:39:522
24564991413,1cyclictest0-21swapper/110:39:111
24564991411,2cyclictest22319-21cups-browsed09:40:231
24564991410,3cyclictest8623-21uname11:00:241
24564991410,2cyclictest28631-21wc09:35:231
2456499140,12cyclictest3602-21grep08:40:141
2456199147,6cyclictest24634-21sh09:33:250
2456199146,4cyclictest225-21systemd-journal10:49:590
2456199141,2cyclictest141rcu_preempt10:32:470
24561991411,2cyclictest7300-21taskset10:20:350
24561991411,2cyclictest13776-21cron07:55:000
24561991410,2cyclictest19064-21wc09:10:200
2456199140,9cyclictest0-21swapper/008:10:150
2456199140,1cyclictest0-21swapper/011:24:050
81852131,1sleep30-21swapper/312:37:253
2456999139,3cyclictest7353-21sed08:45:273
2456999139,3cyclictest23419-21ssh10:30:263
2456999138,3cyclictest11663-21sh10:43:193
24569991312,1cyclictest55550irq/126-enp2s0-09:54:293
24569991312,1cyclictest0-21swapper/311:02:523
24569991310,2cyclictest3957-21cut08:40:163
24569991310,2cyclictest30828-21wc07:20:233
24569991310,2cyclictest28010-21tail09:35:173
24569991310,2cyclictest2435-21head11:55:163
24569991310,2cyclictest1628-21cron07:29:593
24569991310,2cyclictest11696-21tr09:45:133
2456999130,12cyclictest0-21swapper/309:21:083
2456799139,3cyclictest32569-21grep07:25:152
24567991310,2cyclictest6383-21sh07:40:002
24567991310,2cyclictest29274-21cron07:20:002
24567991310,2cyclictest27304-21cron12:30:002
24567991310,2cyclictest27220-21tr12:10:152
24567991310,2cyclictest19981-21cat09:30:222
24567991310,2cyclictest19466-21grep08:05:172
24567991310,2cyclictest18544-21cron08:05:002
24567991310,2cyclictest16113-21sort11:05:152
2456799130,13cyclictest0-21swapper/212:04:302
2456799130,12cyclictest23500-21head10:50:152
2456499139,3cyclictest30327-21grep07:20:181
2456499139,3cyclictest24089-21ssh11:10:151
2456499139,3cyclictest24089-21ssh11:10:141
2456499139,3cyclictest20080-21sh10:10:001
2456499139,3cyclictest0-21swapper/108:00:131
2456499138,2cyclictest23725-21ssh12:07:581
2456499132,10cyclictest12700-21ssh09:45:211
2456499131,8cyclictest225-21systemd-journal09:35:001
24564991312,1cyclictest0-21swapper/111:39:121
24564991312,1cyclictest0-21swapper/110:41:361
24564991310,2cyclictest5832-21sort10:20:181
24564991310,2cyclictest3561-21wc10:00:011
24564991310,2cyclictest27262-21tr08:20:211
24564991310,2cyclictest15165-21grep10:45:171
2456499130,1cyclictest32551-21fschecks_count07:25:151
2456499130,13cyclictest0-21swapper/110:14:181
2456499130,12cyclictest6678-21taskset12:36:111
2456199139,3cyclictest1601-21grep11:15:260
2456199139,3cyclictest11707-21tr09:25:180
2456199137,5cyclictest6045-21apt-get10:40:130
24561991312,1cyclictest0-21swapper/012:23:560
24561991310,2cyclictest8462-21cron11:40:000
24561991310,2cyclictest27705-21grep12:10:170
24561991310,2cyclictest26345-21cron12:10:000
24561991310,2cyclictest18568-21awk08:05:010
2456199130,12cyclictest7414-21tr11:00:160
2456999129,2cyclictest2041-21grep07:30:163
2456999129,2cyclictest16700-21tr08:00:143
2456999129,2cyclictest15768-21sed07:55:263
2456999120,1cyclictest0-21swapper/312:10:493
2456799129,2cyclictest8859-21grep08:50:162
2456799129,2cyclictest6239-21wc10:20:212
2456799129,2cyclictest4328-21grep10:00:152
2456799129,2cyclictest31402-21cut10:35:222
2456799129,2cyclictest31097-21cut08:30:162
2456799129,2cyclictest24854-21grep08:15:212
2456799129,2cyclictest23708-21chrt11:29:342
2456799129,2cyclictest14356-21grep10:45:142
2456799129,2cyclictest13983-21tr09:00:202
2456799129,2cyclictest11945-21tr07:50:142
2456799128,3cyclictest2518-21cat08:35:282
24567991212,0cyclictest0-21swapper/210:31:392
2456799120,11cyclictest4954-21sshd08:40:252
2456499129,2cyclictest7363-21grep08:45:271
2456499129,2cyclictest5787-21cat07:35:271
2456499129,2cyclictest32586-21expr11:15:161
2456499129,2cyclictest26937-21grep07:15:001
2456499129,2cyclictest26699-21ssh11:30:271
2456499129,2cyclictest25935-21grep11:50:131
2456499129,2cyclictest25739-21grep09:15:001
2456499129,2cyclictest24171-21grep08:15:151
2456499129,2cyclictest22728-21sed10:30:191
2456499129,2cyclictest18181-21cut11:45:171
2456499128,3cyclictest10711-21grep08:55:151
2456499127,4cyclictest16951-21ssh11:25:161
2456499125,4cyclictest225-21systemd-journal07:44:591
2456499120,12cyclictest0-21swapper/110:00:301
2456499120,11cyclictest31762-21sed08:30:191
2456199129,2cyclictest26561-21tr09:15:150
2456199129,2cyclictest25328-21grep08:15:260
2456199129,2cyclictest24305-21perf11:30:000
2456199129,2cyclictest20139-21wc08:05:220
2456199129,2cyclictest15226-21wc10:25:260
2456199129,2cyclictest1071-21sh11:55:000
2456199128,3cyclictest9372-21cron12:00:000
2456199125,6cyclictest17346-21diskmemload09:58:540
2456199120,12cyclictest0-21swapper/012:15:470
2456199120,11cyclictest21374-21head12:25:250
139482120,1sleep10-21swapper/112:02:031
2456999118,2cyclictest9028-21ssh11:40:153
2456999118,2cyclictest711-21ssh10:55:333
2456999118,2cyclictest6948-21ssh11:38:493
2456999118,2cyclictest5831-21sed07:35:263
2456999118,2cyclictest5375-21pmu-power10:00:223
2456999118,2cyclictest494-21taskset08:33:083
2456999118,2cyclictest3402-21grep09:40:163
2456999118,2cyclictest32487-21tr11:15:163
2456999118,2cyclictest30913-21grep10:15:243
2456999118,2cyclictest30064-21ssh10:35:123
2456999118,2cyclictest28139-21sed09:15:273
2456999118,2cyclictest27475-21ssh10:14:153
2456999118,2cyclictest26792-21grep08:20:173
2456999118,2cyclictest25801-21cat11:30:183
2456999118,2cyclictest22427-21ssh09:12:243
2456999118,2cyclictest21733-21df_abs08:10:163
2456999118,2cyclictest20533-21chrt12:05:293
2456999118,2cyclictest19190-21rm11:06:483
2456999118,2cyclictest19074-21ssh10:09:143
2456999118,2cyclictest18549-21ssh12:24:433
2456999118,2cyclictest18480-21missed_timers11:45:193
2456999118,2cyclictest17646-21tr11:25:223
2456999118,2cyclictest1710-21sed08:35:183
2456999118,2cyclictest14187-21cut10:25:163
2456999118,2cyclictest11937-21grep07:50:153
2456999118,2cyclictest11353-21sed12:00:243
2456999118,2cyclictest10205-21cat07:45:183
2456999115,5cyclictest22296-21apache212:27:383
2456999115,2cyclictest225-21systemd-journal07:44:253
2456999113,6cyclictest0-21swapper/309:55:173
2456999111,7cyclictest24122-21df_abs08:15:163
2456799119,1cyclictest14504-21expr07:55:142
2456799118,2cyclictest8308-21chrt11:00:212
2456799118,2cyclictest3802-21ssh11:55:352
2456799118,2cyclictest30773-21cat10:15:242
2456799118,2cyclictest29965-21ssh09:55:282
2456799118,2cyclictest28275-21ssh11:50:532
2456799118,2cyclictest25079-21grep11:30:152
2456799118,2cyclictest25070-21ssh10:12:222
2456799118,2cyclictest24968-21sh09:14:242
2456799118,2cyclictest22665-21rm09:51:082
2456799118,2cyclictest16893-21ssh10:26:352
2456799118,2cyclictest11659-21grep09:45:142
2456799118,2cyclictest10903-21grep07:45:262
2456799110,10cyclictest9642-21ssh09:44:022
2456799110,10cyclictest4014-21cat09:20:272
2456499119,1cyclictest41932sleep111:55:541
2456499118,2cyclictest8048-21taskset08:50:071
2456499118,2cyclictest5210-21ssh12:15:491
2456499118,2cyclictest32445-21ssh12:13:151
2456499118,2cyclictest29173-21ssh09:16:091
2456499118,2cyclictest28750-21rm10:53:381
2456499118,2cyclictest20154-21cstates09:50:191
2456499118,2cyclictest17228-21taskset09:08:451
2456499118,2cyclictest1687-21head08:35:191
2456499118,2cyclictest13562-21head09:00:171
2456499118,2cyclictest12437-21ssh09:25:251
2456499118,2cyclictest10990-21ssh12:20:091
2456499117,3cyclictest9466-21grep11:20:211
2456499110,10cyclictest4072-21date07:35:001
2456199118,2cyclictest8213-21ssh12:37:240
2456199118,2cyclictest5003-21grep07:35:190
2456199118,2cyclictest3897-21ssh10:19:370
2456199118,2cyclictest31269-21tr08:30:150
2456199118,2cyclictest30770-21ssh10:55:010
2456199118,2cyclictest30510-21ssh11:14:080
2456199118,2cyclictest30510-21ssh11:14:080
2456199118,2cyclictest29381-21sed08:25:190
2456199118,2cyclictest2684-21head07:30:190
2456199118,2cyclictest24202-21ssh09:52:200
2456199118,2cyclictest12924-21tr09:00:150
2456199118,2cyclictest10209-21awk08:55:000
2456199117,3cyclictest26751-21sed11:30:270
2456199116,4cyclictest1908-21wc08:35:210
2456199116,3cyclictest15667-21sed07:55:250
2456199116,3cyclictest0-21swapper/008:55:140
2456199112,7cyclictest22308-21apache210:10:060
2456999108,1cyclictest9943-21chrt08:51:553
2456999108,1cyclictest28717-21cut08:25:153
2456999108,1cyclictest28106-21uname07:15:193
2456999108,1cyclictest19279-21chrt08:05:153
2456999108,1cyclictest170852sleep309:06:543
2456999107,2cyclictest25525-21tail07:10:213
2456999107,2cyclictest10685-21expr08:55:153
2456999100,9cyclictest13262-21expr09:00:153
2456799108,1cyclictest7398-21expr07:40:172
2456799108,1cyclictest5807-21cut08:45:152
2456799108,1cyclictest30091-21cat08:25:262
2456799108,1cyclictest29828-21expr07:20:162
2456799108,1cyclictest2626-21head07:30:192
2456799108,1cyclictest16998-21seq09:05:472
2456799108,1cyclictest12245-21seq08:56:012
2456799107,2cyclictest26039-21aten2_r7power_c08:20:132
2456799107,2cyclictest24937-21cut07:10:182
2456799107,2cyclictest21839-21grep08:10:182
2456499108,1cyclictest29037-21cron07:16:591
2456499108,1cyclictest28971-21expr08:25:151
2456499108,1cyclictest21634-21cat08:10:161
2456499108,1cyclictest15207-21ntpq07:55:201
2456499107,2cyclictest19306-21sed08:05:161
2456499107,2cyclictest13337-21grep07:50:251
2456199108,1cyclictest28553-21mailstats07:15:220
2456199108,1cyclictest170672sleep009:06:390
2456199107,2cyclictest9663-21ssh12:00:010
2456199107,2cyclictest6482-21sed08:45:180
2456199107,2cyclictest27887-21grep09:35:170
2456199107,2cyclictest26780-21cut08:20:170
2456199100,9cyclictest25594-21which07:10:210
245619997,1cyclictest32242-21ls07:25:140
245619996,2cyclictest30281-21tail07:20:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional