You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 22:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun May 24, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1711029684,8sleep00-21swapper/007:07:230
1716129369,13sleep10-21swapper/107:08:041
1706028976,9sleep20-21swapper/207:06:442
1702428570,9sleep30-21swapper/307:06:163
1740399260,25cyclictest0-21swapper/212:26:072
1740399250,25cyclictest0-21swapper/211:09:022
1740399250,16cyclictest0-21swapper/212:01:242
1740399240,24cyclictest0-21swapper/210:31:332
1740399240,24cyclictest0-21swapper/210:31:322
1740399230,23cyclictest0-21swapper/210:36:402
1740399230,22cyclictest0-21swapper/210:05:122
17401992222,0cyclictest0-21swapper/110:39:261
17396992222,0cyclictest0-21swapper/010:15:480
17406992019,1cyclictest145562sleep309:30:583
1740699200,19cyclictest0-21swapper/311:10:083
1740399200,20cyclictest0-21swapper/212:32:002
17401992017,2cyclictest537-21snmpd12:14:361
1739699200,6cyclictest0-21swapper/012:32:180
17406991917,1cyclictest0-21swapper/310:54:163
1740699191,16cyclictest24066-21ssh10:35:083
1740199198,1cyclictest141rcu_preempt12:33:021
1740199182,2cyclictest0-21swapper/108:40:121
17401991818,0cyclictest0-21swapper/111:21:261
17401991816,1cyclictest537-21snmpd09:39:571
17401991812,3cyclictest0-21swapper/110:30:471
17401991812,3cyclictest0-21swapper/110:30:461
1740199180,2cyclictest10195-21diskmemload11:32:421
30592170,1sleep20-21swapper/211:40:022
1740699173,10cyclictest20309-21wget09:35:133
17403991715,2cyclictest32-21ksoftirqd/210:59:252
1740399170,17cyclictest0-21swapper/211:19:352
1740399170,17cyclictest0-21swapper/210:20:252
1740399170,17cyclictest0-21swapper/209:24:392
1740199174,9cyclictest0-21swapper/108:25:141
1740199170,15cyclictest0-21swapper/108:35:141
17406991614,1cyclictest11538-21ssh10:08:153
17406991611,3cyclictest3748-21chrt11:20:213
1740399162,1cyclictest225-21systemd-journal09:49:592
1740399160,16cyclictest0-21swapper/212:14:532
17401991614,1cyclictest0-21swapper/109:55:491
17396991615,1cyclictest0-21swapper/011:11:360
1740699151,4cyclictest225-21systemd-journal11:39:543
1740399154,9cyclictest0-21swapper/210:15:032
1740399152,10cyclictest55650irq/127-enp2s0-10:45:132
1740399150,3cyclictest0-21swapper/211:28:232
17401991515,0cyclictest0-21swapper/110:10:291
17401991515,0cyclictest0-21swapper/109:53:071
17401991514,1cyclictest0-21swapper/109:23:421
1740199150,15cyclictest0-21swapper/112:09:181
1739699158,6cyclictest18778-21awk08:19:590
17396991515,0cyclictest0-21swapper/011:37:460
17396991513,1cyclictest0-21swapper/012:26:080
17396991510,5cyclictest4854-21apache210:39:180
1740699145,5cyclictest225-21systemd-journal09:20:013
17406991411,2cyclictest7989-21chrt12:01:363
1740399146,3cyclictest22157-21munin-run10:15:012
1740399145,7cyclictest2355-21apt-get11:20:122
17403991414,0cyclictest0-21swapper/210:26:172
17403991411,2cyclictest13945-21wc12:05:192
1740399140,14cyclictest0-21swapper/211:58:422
1740399140,14cyclictest0-21swapper/209:44:482
1740399140,14cyclictest0-21swapper/209:38:392
17401991414,0cyclictest0-21swapper/112:18:071
17401991414,0cyclictest0-21swapper/110:27:191
17401991413,1cyclictest0-21swapper/112:27:441
17401991410,2cyclictest0-21swapper/112:39:191
17396991411,2cyclictest1605-21cron07:45:000
1740699139,3cyclictest4161-21cat09:45:003
1740699139,3cyclictest20817-21rm11:50:173
1740699139,2cyclictest5455-21basename09:00:003
1740699133,6cyclictest3925-21df_abs11:40:153
17406991310,2cyclictest8756-21uniq09:05:173
17406991310,2cyclictest32039-21basename07:40:133
17406991310,2cyclictest26403-21grep08:35:143
17406991310,2cyclictest22195-21head08:25:183
1740399139,3cyclictest19901-21wc11:10:262
1740399139,3cyclictest19309-21grep09:15:152
17403991310,2cyclictest7808-21cron09:05:002
17403991310,2cyclictest4935-21sed08:55:262
17403991310,2cyclictest19842-21wc09:35:002
17403991310,2cyclictest18260-21grep08:15:262
1740399130,1cyclictest0-21swapper/212:20:142
1740399130,1cyclictest0-21swapper/212:20:132
1740399130,13cyclictest0-21swapper/211:37:592
1740199139,4cyclictest4844-21apache211:41:481
1740199139,3cyclictest8950-21wc10:45:151
1740199139,2cyclictest7420-21grep07:55:171
1740199139,2cyclictest29454-21head09:40:171
1740199136,0cyclictest0-21swapper/110:06:331
17401991311,1cyclictest0-21swapper/111:05:331
17401991310,2cyclictest17353-21tail08:15:171
1740199130,4cyclictest8413-21sh09:27:421
1740199130,12cyclictest31264-21grep10:00:211
1740199130,12cyclictest25459-21cut10:55:141
1739699139,3cyclictest9106-21cat09:05:200
1739699131,3cyclictest0-21swapper/009:15:130
17396991313,0cyclictest0-21swapper/007:10:180
17396991311,2cyclictest13-21ksoftirqd/010:11:520
17396991310,2cyclictest27954-21cron11:54:590
17396991310,2cyclictest24868-21cat10:55:000
17396991310,2cyclictest2093-21ls08:50:220
17396991310,2cyclictest2093-21ls08:50:210
17396991310,2cyclictest20741-21head07:15:160
1739699130,1cyclictest0-21swapper/010:55:580
1739699130,12cyclictest32014-21ssh12:15:390
1739699130,0cyclictest0-21swapper/011:27:430
1740699129,2cyclictest31103-21grep10:00:193
1740699129,2cyclictest28946-21sh11:55:143
1740699129,2cyclictest27273-21grep10:55:253
1740699129,2cyclictest23421-21sed09:55:263
1740699129,2cyclictest22875-21ssh12:10:273
1740699129,2cyclictest22440-21grep07:20:133
1740699129,2cyclictest15937-21tr10:30:153
1740699129,2cyclictest15937-21tr10:30:143
1740699129,2cyclictest14036-21ssh10:10:033
1740699129,2cyclictest13645-21grep12:25:013
1740699129,2cyclictest13645-21grep12:25:003
1740699129,2cyclictest12615-21tail11:45:173
1740699128,3cyclictest22876-21ssh12:30:153
1740699128,3cyclictest2074-21tail10:40:263
1740699128,2cyclictest25913-21ntpq07:25:213
1740699127,2cyclictest3647-21wc09:25:153
1740699124,7cyclictest455-21dbus-daemon08:00:563
1740699121,8cyclictest19070-21/usr/sbin/munin11:30:153
1740699120,11cyclictest29440-21sed09:20:263
1740699120,11cyclictest2028-21grep08:50:213
1740699120,11cyclictest2028-21grep08:50:203
1740399129,2cyclictest3587-21cut09:25:152
1740399129,2cyclictest31700-21grep12:15:262
1740399129,2cyclictest26420-21grep07:25:272
1740399129,2cyclictest2594-21grep08:50:272
1740399129,2cyclictest2594-21grep08:50:262
1740399129,2cyclictest21292-21wc11:50:202
1740399129,2cyclictest15906-21grep08:10:272
1740399128,3cyclictest8216-21wc10:45:012
1740399128,3cyclictest5795-21wc07:50:262
1740399128,3cyclictest31256-21tr10:00:212
1740399128,3cyclictest22076-21cut08:25:172
1740399128,3cyclictest21459-21ssh09:55:112
1740399127,3cyclictest28777-21tr08:40:142
1740199129,2cyclictest5298-21head12:00:161
1740199129,2cyclictest4052-21cat07:49:591
1740199129,2cyclictest31351-21cut10:20:131
1740199129,2cyclictest3039-21sh11:39:591
1740199129,2cyclictest24980-21grep07:25:151
1740199129,2cyclictest19957-21grep09:15:171
1740199129,2cyclictest11536-21cut09:10:161
1740199128,3cyclictest27552-21ssh11:15:201
1740199127,4cyclictest15187-21uname08:10:191
1740199123,7cyclictest0-21swapper/108:55:151
17401991211,1cyclictest0-21swapper/112:24:521
17401991211,1cyclictest0-21swapper/112:24:521
17401991210,2cyclictest0-21swapper/111:51:211
17401991210,1cyclictest5517-21grep09:45:161
17401991210,1cyclictest29270-21ssh11:55:181
1739699129,2cyclictest6765-21grep10:05:170
1739699129,2cyclictest6464-21cat09:45:240
1739699129,2cyclictest32069-21grep12:35:210
1739699129,2cyclictest31821-21sed10:00:250
1739699129,2cyclictest20507-21sed11:30:220
1739699129,2cyclictest10810-21ssh11:05:210
1739699129,2cyclictest10322-21cat08:00:200
1739699128,3cyclictest21210-21cat08:25:000
1739699121,8cyclictest29913-21cut09:40:200
17396991210,1cyclictest6330-21grep09:00:170
17396991210,1cyclictest32048-21tr08:45:200
1740699119,2cyclictest0-21swapper/307:15:133
1740699119,1cyclictest17672-21grep07:10:193
1740699118,2cyclictest8748-21chrt09:47:003
1740699118,2cyclictest8651-21cut10:25:223
1740699118,2cyclictest7765-21taskset09:04:343
1740699118,2cyclictest720-21grep10:20:263
1740699118,2cyclictest29263-21latency_hist07:34:593
1740699118,2cyclictest2714-21cat07:45:173
1740699118,2cyclictest26151-21ssh11:15:023
1740699118,2cyclictest1863-21ssh12:17:263
1740699118,2cyclictest1630-21grep11:00:173
1740699118,2cyclictest15015-21head08:10:173
1740699118,2cyclictest14360-21tr12:25:163
1740699118,2cyclictest14144-21chrt11:26:313
1740699118,2cyclictest14000-21sort12:05:203
1740699118,2cyclictest11790-21grep09:10:183
1740699118,2cyclictest11452-21ssh10:46:063
1740699118,2cyclictest11392-21smartctl11:05:273
1740699117,2cyclictest19029-21apache_accesses08:20:123
1740699116,3cyclictest1-21systemd08:44:193
1740699111,8cyclictest39-21ksoftirqd/308:05:143
1740699110,6cyclictest0-21swapper/308:45:123
1740699110,10cyclictest13216-21cut09:50:143
1740399118,2cyclictest9294-21chrt08:00:142
1740399118,2cyclictest31329-21fschecks_count12:35:152
1740399118,2cyclictest29066-21chrt07:32:362
1740399118,2cyclictest25482-21head08:30:252
1740399118,2cyclictest24016-21sed07:20:252
1740399118,2cyclictest19330-21head07:10:322
1740399118,2cyclictest19012-21sh10:50:292
1740399118,2cyclictest16817-21ssh09:51:462
1740399118,2cyclictest1415-21chrt07:42:392
1740399118,2cyclictest14063-21rm09:11:282
1740399118,2cyclictest12079-21tr11:45:142
1740399118,2cyclictest11916-21grep08:05:142
1740399117,3cyclictest27188-21wc08:35:192
1740399117,2cyclictest31152-21tr08:45:152
1740399114,3cyclictest225-21systemd-journal11:30:002
1740399110,1cyclictest0-21swapper/211:04:362
1740399110,10cyclictest0-21swapper/207:35:142
1740199119,1cyclictest10195-21diskmemload11:10:241
1740199118,2cyclictest855-21sort10:40:171
1740199118,2cyclictest5926-21sort09:00:151
1740199118,2cyclictest458-21taskset08:48:401
1740199118,2cyclictest31615-21chrt07:39:531
1740199118,2cyclictest29808-21chrt10:19:171
1740199118,2cyclictest24211-21cat08:30:131
1740199118,2cyclictest2134-21ssh11:00:181
1740199118,2cyclictest20079-21ssh10:51:191
1740199118,2cyclictest19659-21sort08:20:171
1740199118,2cyclictest18540-21ssh09:34:041
1740199118,2cyclictest12916-21ssh11:25:321
1740199117,3cyclictest17152-21ssh11:48:041
1739699119,1cyclictest27802-21grep07:30:160
1739699119,1cyclictest22394-21sed09:55:180
1739699118,2cyclictest993-21tail10:40:160
1739699118,2cyclictest8971-21date07:59:590
1739699118,2cyclictest5083-21head07:50:190
1739699118,2cyclictest4586-21wc09:25:190
1739699118,2cyclictest4299-21ssh11:01:130
1739699118,2cyclictest23924-21grep08:30:120
1739699118,2cyclictest21661-21grep08:25:160
1739699118,2cyclictest1829-21awk11:19:590
1739699118,2cyclictest13423-21tr09:50:130
1739699118,2cyclictest1325-21ssh10:20:470
1739699118,2cyclictest12664-21tail11:45:180
1739699118,2cyclictest11768-21ssh09:30:090
1739699118,2cyclictest0-21swapper/012:20:140
1739699118,2cyclictest0-21swapper/012:20:130
1739699113,7cyclictest10195-21diskmemload11:22:350
1739699111,2cyclictest0-21swapper/010:32:070
1739699111,2cyclictest0-21swapper/010:32:070
1740699108,1cyclictest30926-21sh07:35:253
1740699108,1cyclictest30660-21chrt12:35:103
1740699108,1cyclictest25141-21mailstats08:30:213
1740699108,1cyclictest23555-21ssh10:15:153
1740699108,1cyclictest17105-21expr08:15:153
17406991010,0cyclictest0-21swapper/307:50:143
1740699100,9cyclictest7843-21ntp_states07:55:223
1740399108,1cyclictest8175-21cut09:05:142
1740399108,1cyclictest3132-21grep07:45:212
1740399107,2cyclictest8460-21seq07:55:552
1740399107,2cyclictest20733-21grep07:15:162
1740399107,2cyclictest19937-21cat08:20:192
1740199108,1cyclictest21884-21taskset07:17:081
1740199108,1cyclictest11914-21expr08:05:141
1740199107,2cyclictest32061-21cut07:40:141
1740199107,2cyclictest27140-21cat07:30:101
1740199100,9cyclictest27202sleep108:51:451
1740199100,9cyclictest27202sleep108:51:441
1740199100,9cyclictest17653-21hostname07:10:181
1739699108,2cyclictest9688-21latency10:45:200
1739699108,2cyclictest14632-21expr08:10:150
1739699108,1cyclictest7113-21ssh10:25:060
1739699108,1cyclictest30396-21grep11:55:280
1739699108,1cyclictest27353-21ssh09:20:070
1739699108,1cyclictest2520-21cut07:45:150
1739699108,1cyclictest10699-21ssh09:10:120
1739699107,3cyclictest28236-21taskset08:39:480
1739699107,2cyclictest6449-21grep12:00:280
1739699107,2cyclictest29307-21head08:40:160
1739699107,2cyclictest24865-21sh09:37:270
1739699107,2cyclictest24436-21chrt07:24:080
1739699107,2cyclictest20947-21ssh12:10:030
1739699107,2cyclictest12776-21head08:05:190
1739699100,9cyclictest31532sleep011:40:020
1739699100,9cyclictest12782-21sh12:05:110
174019997,1cyclictest9675-21expr08:00:161
174019997,1cyclictest8695-21tune2fs09:05:161
174019996,2cyclictest5951-21sed07:50:281
174019991,4cyclictest225-21systemd-journal07:23:171
173969997,2cyclictest4649-21python08:55:260
173969997,1cyclictest29990-21ls07:35:140
173969987,1cyclictest25542-21sed07:25:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional