You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 14:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Jul 22, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1708429178,9sleep00-21swapper/007:06:090
1729928764,18sleep20-21swapper/207:08:572
1737028669,13sleep10-21swapper/107:09:521
1722328673,9sleep30-21swapper/307:07:573
60342690,0sleep00-21swapper/009:25:530
1746399287,20cyclictest10211-21diskmemload09:53:461
1745699269,16cyclictest0-21swapper/007:30:130
1745699258,14cyclictest0-21swapper/010:18:400
17456992510,11cyclictest0-21swapper/012:31:420
17472992420,3cyclictest19809-21apt-get07:15:023
17456992422,1cyclictest10211-21diskmemload11:26:020
1747299230,21cyclictest0-21swapper/311:02:033
1746799230,22cyclictest0-21swapper/209:45:142
17456992216,5cyclictest10211-21diskmemload09:41:120
17456992211,10cyclictest0-21swapper/012:35:130
1745699220,8cyclictest0-21swapper/009:37:280
17472992119,0cyclictest0-21swapper/309:29:513
1747299210,3cyclictest0-21swapper/309:45:313
17456992113,6cyclictest0-21swapper/012:14:320
1745699207,12cyclictest16722-21apache209:51:430
17456992019,1cyclictest0-21swapper/007:35:150
17456992019,1cyclictest0-21swapper/007:35:140
17456992017,2cyclictest0-21swapper/011:00:380
17456992015,4cyclictest0-21swapper/009:00:170
17456992015,2cyclictest10211-21diskmemload10:05:130
17456992012,7cyclictest0-21swapper/011:17:310
17456992010,8cyclictest0-21swapper/011:57:130
1746399192,10cyclictest0-21swapper/111:52:481
17456991917,1cyclictest10211-21diskmemload10:40:210
17456991916,2cyclictest10211-21diskmemload12:15:230
17456991912,6cyclictest0-21swapper/011:22:200
1746799180,18cyclictest0-21swapper/211:55:092
1746799180,18cyclictest0-21swapper/211:37:062
1746399183,1cyclictest27245-21sed11:15:151
1746399180,3cyclictest0-21swapper/111:27:411
1745699188,9cyclictest0-21swapper/012:02:160
1745699188,9cyclictest0-21swapper/010:29:210
1745699187,7cyclictest22174-21grep09:55:180
1745699186,9cyclictest0-21swapper/011:10:410
1745699185,10cyclictest0-21swapper/012:25:130
1745699181,3cyclictest0-21swapper/012:05:150
17456991812,5cyclictest0-21swapper/009:46:230
1747299178,2cyclictest10651-21/usr/sbin/munin09:10:093
1746799176,8cyclictest0-21swapper/212:24:372
1746799170,16cyclictest0-21swapper/208:25:142
1745699179,6cyclictest0-21swapper/011:44:590
1745699177,8cyclictest0-21swapper/009:30:260
1745699177,8cyclictest0-21swapper/009:30:250
1745699175,11cyclictest0-21swapper/009:14:380
1745699172,4cyclictest0-21swapper/010:20:440
17456991715,1cyclictest10211-21diskmemload11:09:550
17456991714,2cyclictest22506-21wc08:25:200
17456991714,2cyclictest0-21swapper/010:01:130
17456991710,3cyclictest0-21swapper/010:45:510
1745699170,12cyclictest0-21swapper/008:45:150
17472991615,1cyclictest7452-21kworker/u8:011:51:413
17467991612,3cyclictest28859-21ls07:30:232
1746399161,4cyclictest0-21swapper/110:22:521
17463991614,1cyclictest16781-21ssh10:50:031
1746399160,16cyclictest0-21swapper/112:07:501
1746399160,15cyclictest24721-21munin-node10:05:131
1745699162,11cyclictest0-21swapper/011:47:540
17456991613,2cyclictest14119-21cron12:24:590
17456991611,2cyclictest0-21swapper/011:33:580
17456991611,2cyclictest0-21swapper/010:39:110
17472991514,1cyclictest0-21swapper/310:04:033
17472991513,2cyclictest0-21swapper/308:35:133
17467991514,1cyclictest55850irq/129-enp2s0-10:15:212
1746399154,4cyclictest225-21systemd-journal08:00:011
1746399151,4cyclictest141rcu_preempt10:57:391
17463991513,2cyclictest25-21ksoftirqd/109:30:251
17463991513,2cyclictest25-21ksoftirqd/109:30:241
1746399151,13cyclictest10211-21diskmemload11:39:431
1746399150,13cyclictest0-21swapper/110:35:151
1745699158,5cyclictest0-21swapper/007:20:250
1745699156,7cyclictest16093-21cstates10:30:140
17456991514,1cyclictest0-21swapper/011:52:300
200632140,1sleep20-21swapper/209:35:062
1747299142,7cyclictest0-21swapper/308:30:153
1747299140,11cyclictest0-21swapper/312:30:133
17467991410,3cyclictest22663-21tr09:55:212
1746799140,14cyclictest0-21swapper/210:08:292
1746799140,13cyclictest2966-21ssh09:25:002
1746799140,10cyclictest0-21swapper/210:55:142
1746399148,5cyclictest8717-21ntpq10:25:221
17463991412,1cyclictest6883-21awk09:00:221
1746399140,14cyclictest0-21swapper/110:46:141
1746399140,13cyclictest0-21swapper/112:30:281
1745699148,5cyclictest15698-21awk08:10:270
1745699147,6cyclictest0-21swapper/009:16:400
1745699147,6cyclictest0-21swapper/007:15:160
1745699143,2cyclictest0-21swapper/008:20:140
1745699143,10cyclictest25433-21rm10:55:130
1745699142,4cyclictest19939-21ssh10:51:090
17456991413,1cyclictest0-21swapper/009:20:570
1747299132,10cyclictest0-21swapper/309:44:343
17472991311,2cyclictest39-21ksoftirqd/312:00:083
17472991310,2cyclictest23695-21basename10:35:003
17472991310,2cyclictest16582-21head12:25:253
17472991310,2cyclictest14187-21ssh11:45:283
17472991310,2cyclictest11626-21cat11:05:253
17472991310,2cyclictest11416-21awk08:05:003
1746799138,5cyclictest16722-21apache211:46:452
1746799133,9cyclictest25018-21runrttasks09:18:282
17467991310,2cyclictest23809-21tr07:20:192
17467991310,2cyclictest14313-21grep08:10:162
1746799130,12cyclictest27322-21turbostat11:35:002
1746399139,3cyclictest4311-21cat11:20:241
1746399139,3cyclictest26760-21head07:25:271
1746399139,3cyclictest20119-21ssh11:30:151
1746399139,3cyclictest17711-21grep10:30:251
1746399139,3cyclictest15675-21sed08:10:271
1746399138,3cyclictest1269-21wc07:40:251
1746399133,5cyclictest225-21systemd-journal09:25:001
17463991312,1cyclictest0-21swapper/112:15:031
17463991310,2cyclictest4608-21sed11:40:161
17463991310,2cyclictest22225-21cut09:55:181
17463991310,2cyclictest22143-21grep12:10:141
17463991310,2cyclictest22070-21wc09:35:271
17463991310,2cyclictest16652-21tr08:15:151
17463991310,2cyclictest10193-21cron09:10:001
1746399130,4cyclictest22590-21needreboot08:25:201
1746399130,1cyclictest0-21swapper/111:59:151
1746399130,13cyclictest0-21swapper/110:41:291
1745699139,4cyclictest0-21swapper/011:35:160
1745699139,3cyclictest4260-21cron07:50:010
17456991310,2cyclictest26966-21grep08:35:170
17456991310,2cyclictest14922-21ssh10:10:150
17456991310,2cyclictest13307-21grep08:05:260
17456991310,2cyclictest10331-21wc08:00:210
1747299129,2cyclictest918-21ssh10:40:183
1747299129,2cyclictest6450-21grep09:00:183
1747299129,2cyclictest4514-21grep11:40:163
1747299129,2cyclictest31947-21cut08:45:193
1747299129,2cyclictest30342-21tr11:55:223
1747299129,2cyclictest29769-21turbostat.cron12:15:003
1747299129,2cyclictest27711-21ssh11:15:183
1747299129,2cyclictest22232-21sed09:35:283
1747299129,2cyclictest18021-21ssh09:33:403
1747299129,2cyclictest18021-21ssh09:33:403
1747299129,2cyclictest16224-21cron10:50:013
1747299129,2cyclictest14842-21head09:50:243
1747299129,2cyclictest14768-21grep08:10:173
1747299128,3cyclictest3964-21rm10:22:543
17472991211,1cyclictest0-21swapper/312:15:143
1747299120,11cyclictest0-21swapper/309:57:453
1746799129,2cyclictest9198-21expr10:45:172
1746799129,2cyclictest7694-21sed07:55:212
1746799129,2cyclictest5954-21tr09:00:162
1746799129,2cyclictest31916-21cron07:40:012
1746799129,2cyclictest31916-21cron07:40:002
1746799129,2cyclictest31700-21tr08:45:172
1746799129,2cyclictest2874-21grep07:45:182
1746799129,2cyclictest18409-21grep07:10:152
1746799129,2cyclictest12532-21grep09:30:172
1746799129,2cyclictest12532-21grep09:30:172
1746799129,2cyclictest1233-21tr08:50:132
1746799128,3cyclictest13301-21seq11:25:382
1746799127,4cyclictest4540-21apt-get07:50:142
17467991211,1cyclictest0-21swapper/209:41:482
17467991210,1cyclictest23949-21cut08:30:152
1746799120,1cyclictest0-21swapper/209:51:252
1746399129,2cyclictest7056-21head12:20:161
1746399129,2cyclictest6631-21wc12:00:251
1746399129,2cyclictest32569-21wc08:45:261
1746399129,2cyclictest30562-21grep07:35:181
1746399129,2cyclictest30562-21grep07:35:171
1746399129,2cyclictest30002-21tr09:40:201
1746399129,2cyclictest27941-21head08:35:261
1746399129,2cyclictest25-21ksoftirqd/111:03:331
1746399129,2cyclictest11429-21perf08:05:001
1746399128,3cyclictest16363-21wc12:25:261
1746399125,3cyclictest13755-21seq08:09:441
1746399123,7cyclictest24721-21munin-node10:10:071
17463991210,1cyclictest4029-21grep08:55:181
17463991210,1cyclictest30435-21fschecks_count10:00:181
1745699129,2cyclictest9705-21sed09:05:280
1745699129,2cyclictest8620-21grep07:55:290
1745699129,2cyclictest3549-21tr08:55:170
1745699129,2cyclictest27246-21wc07:30:000
1745699129,2cyclictest1358-21wc07:40:260
1745699126,3cyclictest0-21swapper/007:10:120
1747299119,1cyclictest27362-21grep07:30:013
1747299119,1cyclictest23174-21grep10:15:143
1747299118,2cyclictest7188-21grep07:55:183
1747299118,2cyclictest3161-21df11:20:153
1747299118,2cyclictest29738-21ssh09:20:363
1747299118,2cyclictest28890-21chrt10:56:373
1747299118,2cyclictest27675-21ssh11:35:093
1747299118,2cyclictest22224-21grep08:25:173
1747299118,2cyclictest21066-21seq08:23:153
1747299118,2cyclictest19432-21grep09:15:163
1747299118,2cyclictest19083-21ssh10:50:283
1747299118,2cyclictest16815-21ssh10:10:513
1747299118,2cyclictest14898-21head12:05:233
1747299118,2cyclictest13321-21sed08:05:273
1747299118,2cyclictest13238-21ssh12:24:193
1747299118,2cyclictest12581-21ssh10:09:063
1747299118,2cyclictest11307-21grep11:25:153
1747299117,3cyclictest6632-21cron07:55:003
1747299117,3cyclictest31137-21ssh12:35:023
1747299116,4cyclictest10211-21diskmemload11:14:093
1747299115,4cyclictest7428-21apt-get10:25:143
1747299110,10cyclictest32485-21grep07:40:153
1746799119,1cyclictest5937-21head12:00:192
1746799119,1cyclictest25539-21chrt12:30:482
1746799119,1cyclictest1677-21cut11:00:152
1746799119,1cyclictest15445-21grep12:25:172
1746799118,2cyclictest9701-21expr08:00:152
1746799118,2cyclictest8642-21df09:05:162
1746799118,2cyclictest8147-21fschecks_time10:25:182
1746799118,2cyclictest7095-21ssh11:41:282
1746799118,2cyclictest4339-21grep09:25:202
1746799118,2cyclictest3552-21basename11:20:172
1746799118,2cyclictest32239-21sed12:15:272
1746799118,2cyclictest31731-21cat12:35:152
1746799118,2cyclictest3141-21sh10:22:152
1746799118,2cyclictest30275-21sed08:40:262
1746799118,2cyclictest28330-21cron08:39:592
1746799118,2cyclictest27091-21expr11:15:142
1746799118,2cyclictest24956-21grep10:35:172
1746799118,2cyclictest21284-21sh10:33:092
1746799118,2cyclictest21244-21ssh11:50:182
1746799118,2cyclictest20886-21forks07:15:182
1746799118,2cyclictest19670-21rm10:50:552
1746799118,2cyclictest16682-21sh12:06:222
1746799118,2cyclictest16565-21wget08:15:142
1746799118,2cyclictest14162-21rm10:10:112
1746799118,2cyclictest13802-21sh11:06:562
1746799118,2cyclictest13181-21ssh09:10:482
1746799117,3cyclictest42512sleep210:42:082
1746799110,10cyclictest2554-21sh10:02:592
1746799110,10cyclictest25442-21sh12:11:402
1746399118,2cyclictest5897-21ssh09:45:211
1746399118,2cyclictest5045-21tr09:25:251
1746399118,2cyclictest4806-21tr07:50:151
1746399118,2cyclictest4221-21ssh12:38:001
1746399118,2cyclictest29350-21head08:40:171
1746399118,2cyclictest28153-21grep07:30:151
1746399118,2cyclictest22975-21rm09:16:531
1746399118,2cyclictest18992-21sort11:10:171
1746399118,2cyclictest15582-21ssh09:12:401
1746399118,2cyclictest13475-21sh11:06:411
1746399116,3cyclictest23945-21wc10:15:191
17463991110,1cyclictest13658-21mailstats11:45:231
1745699118,2cyclictest1920-21cat08:50:160
1745699118,2cyclictest18190-21head08:15:260
1747299108,2cyclictest21620-21sh11:30:363
1747299108,2cyclictest19295-21ls07:10:243
1747299108,1cyclictest8930-21cat09:05:183
1747299108,1cyclictest4046-21head08:55:183
1747299108,1cyclictest28887-21grep08:40:143
1747299108,1cyclictest16864-21expr08:15:153
1747299108,1cyclictest1279-21expr08:50:133
1747299107,2cyclictest3973-21taskset07:46:233
1747299107,2cyclictest30431-21tune2fs07:35:173
1747299107,2cyclictest30431-21tune2fs07:35:163
1747299107,2cyclictest27365-21ssh10:36:233
1747299107,2cyclictest24330-21sed07:20:253
1746799108,2cyclictest4122-21smartctl08:55:182
1746799108,1cyclictest26663-21date07:25:272
1746799108,1cyclictest20422-21rm11:10:322
1746799108,1cyclictest19544-21expr08:20:162
1746799108,1cyclictest18832sleep207:44:572
1746799108,1cyclictest13597-21taskset08:07:422
1746399108,1cyclictest21404-21ls08:25:001
1746399108,1cyclictest19499-21taskset07:11:121
1746399107,2cyclictest24388-21grep07:20:251
1746399107,2cyclictest24099-21tr08:30:161
1746399107,2cyclictest22056-21taskset07:15:431
1746399107,2cyclictest2126-21taskset07:45:081
1745699108,1cyclictest4560-21cut07:50:140
1745699107,2cyclictest29343-21tail08:40:160
1745699107,2cyclictest24571-21tail08:30:180
174729994,2cyclictest225-21systemd-journal07:35:003
174639997,1cyclictest1755-21sed08:50:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional