You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-21 - 03:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 21, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2507829179,8sleep00-21swapper/019:06:060
2519129064,8sleep20-21swapper/219:07:332
2522228961,9sleep30-21swapper/319:07:593
2527728366,13sleep10-21swapper/119:08:401
2546999260,26cyclictest0-21swapper/100:28:351
2547499242,15cyclictest1651-21ssh23:15:193
2547299212,4cyclictest0-21swapper/221:26:212
2546699211,1cyclictest141rcu_preempt22:47:260
2546999200,20cyclictest0-21swapper/123:58:361
25474991918,1cyclictest18268-21diskmemload22:10:583
2546999193,15cyclictest0-21swapper/123:29:331
2546999190,16cyclictest0-21swapper/100:12:251
25466991918,1cyclictest0-21swapper/000:19:010
2547299188,3cyclictest32-21ksoftirqd/221:35:232
2547299181,16cyclictest30620-21sh22:34:472
2547499170,2cyclictest0-21swapper/321:16:043
25472991714,3cyclictest5603-21latency_hist22:20:012
25469991716,1cyclictest0-21swapper/122:30:141
2546999171,16cyclictest12664-21ssh23:02:011
2546999170,1cyclictest0-21swapper/121:40:151
2546699170,2cyclictest0-21swapper/000:02:500
2547499161,3cyclictest141rcu_preempt19:25:163
2547499161,1cyclictest0-21swapper/323:56:193
25472991611,1cyclictest32-21ksoftirqd/221:30:382
2547299160,16cyclictest0-21swapper/222:45:462
2547299160,15cyclictest0-21swapper/223:11:142
2546999160,1cyclictest0-21swapper/123:51:191
2546999160,16cyclictest0-21swapper/122:10:161
2547499157,4cyclictest39-21ksoftirqd/323:45:213
2547499157,4cyclictest39-21ksoftirqd/323:45:203
25474991515,0cyclictest0-21swapper/323:52:293
25474991514,1cyclictest0-21swapper/323:26:093
2547499150,10cyclictest6062-21cpuspeed_turbos00:35:153
2547499150,10cyclictest0-21swapper/320:05:153
2547299159,1cyclictest32-21ksoftirqd/222:53:412
2547299156,8cyclictest21822-21/usr/sbin/munin22:10:182
25472991513,1cyclictest0-21swapper/223:43:012
25472991511,3cyclictest23643-31find00:07:092
2547299150,14cyclictest0-21swapper/200:24:432
2547299150,13cyclictest0-21swapper/223:49:352
2547299150,13cyclictest0-21swapper/223:49:342
2546999153,6cyclictest18268-21diskmemload21:35:231
25469991515,0cyclictest0-21swapper/121:11:051
2546999150,1cyclictest0-21swapper/122:43:531
2546999150,15cyclictest0-21swapper/100:18:121
2546999150,14cyclictest0-21swapper/122:19:301
2546699150,1cyclictest0-21swapper/023:18:550
2547499148,4cyclictest18268-21diskmemload00:12:513
2547499146,2cyclictest12447-21apt-get21:45:143
2547499140,4cyclictest13119-21kworker/u8:223:42:373
2547299140,2cyclictest0-21swapper/200:03:342
2547299140,13cyclictest0-21swapper/222:44:292
2547299140,13cyclictest0-21swapper/222:25:422
2546999149,4cyclictest31383-21munin-run20:30:001
2546999149,2cyclictest18831-21ssh23:05:261
25469991410,3cyclictest18268-21diskmemload00:39:051
25469991410,3cyclictest18268-21diskmemload00:33:421
2546999140,4cyclictest0-21swapper/123:35:161
2546999140,3cyclictest0-21swapper/123:44:421
2546699140,13cyclictest0-21swapper/022:19:540
2547499139,1cyclictest18268-21diskmemload00:17:013
2547499135,2cyclictest0-21swapper/322:47:283
2547499133,8cyclictest16479-21ssh00:21:323
2547499132,3cyclictest0-21swapper/321:00:113
25474991312,1cyclictest0-21swapper/322:25:143
2547299139,3cyclictest18843-21head20:00:252
2547299138,4cyclictest6508-21grep00:35:172
2547299135,7cyclictest7482-21kernelversion22:20:192
2547299134,7cyclictest1436-21sendmail-mta22:05:132
2547299133,6cyclictest0-21swapper/221:50:162
25472991312,1cyclictest0-21swapper/223:08:192
25472991311,2cyclictest19912-21wc21:10:202
25472991311,1cyclictest9292-21df23:00:182
25472991311,1cyclictest29590-21wc19:15:252
25472991310,2cyclictest6620-21tr20:45:142
25472991310,2cyclictest5488-21sort00:15:192
25472991310,2cyclictest32-21ksoftirqd/221:42:262
25472991310,2cyclictest32-21ksoftirqd/200:10:132
25472991310,2cyclictest29326-21basename00:29:592
25472991310,2cyclictest15004-21tr19:55:152
2547299130,13cyclictest0-21swapper/223:50:322
2546999139,3cyclictest17957-21ssh00:22:431
2546999134,4cyclictest21898-21apt-get20:10:141
2546999134,4cyclictest21898-21apt-get20:10:131
2546999133,8cyclictest18268-21diskmemload22:05:171
25469991310,2cyclictest6928-21taskset23:18:081
25469991310,2cyclictest17421-21grep20:00:141
2546699139,3cyclictest25016-21tr20:15:170
2546699134,2cyclictest18268-21diskmemload00:35:130
2546699131,12cyclictest0-21swapper/021:20:420
25466991311,1cyclictest0-21swapper/021:41:590
25466991310,2cyclictest747-21cat20:30:280
25466991310,2cyclictest20753-21grep21:10:270
25466991310,2cyclictest11090-21sh23:20:180
2546699130,1cyclictest0-21swapper/019:55:130
2546699130,12cyclictest27994-21grep23:30:210
2547499129,2cyclictest991-21grep22:35:273
2547499129,2cyclictest6399-21tr22:00:213
2547499129,2cyclictest6399-21tr22:00:213
2547499129,2cyclictest6273-21sh22:20:103
2547499129,2cyclictest5952-21cut19:35:173
2547499129,2cyclictest28720-21grep19:15:163
2547499129,2cyclictest26556-21wc20:20:013
2547499129,2cyclictest26055-21sed22:50:273
2547499129,2cyclictest25503-21chrt22:30:453
2547499129,2cyclictest21690-21tr00:25:123
2547499129,2cyclictest18268-21diskmemload00:06:143
2547499129,2cyclictest15601-21ssh23:23:203
2547499129,2cyclictest14279-21chrt19:52:333
2547499128,3cyclictest14440-21sh21:26:123
2547499128,2cyclictest24500-21sh23:09:353
2547499124,4cyclictest18268-21diskmemload22:42:223
2547499122,5cyclictest3212-21/usr/sbin/munin23:35:193
2547499121,5cyclictest30223-21apt-get22:15:153
25474991210,1cyclictest9902-21wc23:00:233
2547499120,11cyclictest5742-21cat21:40:253
2547499120,11cyclictest32713-21sh23:33:393
2547299129,2cyclictest8360-21head19:40:172
2547299129,2cyclictest4739-21cut20:40:172
2547299129,2cyclictest3567-21sed19:30:192
2547299129,2cyclictest31707-21grep20:30:162
2547299129,2cyclictest31223-21grep00:30:222
2547299129,2cyclictest29893-21cut21:55:172
2547299129,2cyclictest27999-21sed21:15:162
2547299129,2cyclictest21211-21grep20:05:262
2547299129,2cyclictest19010-21ssh23:25:172
2547299129,2cyclictest15189-21tail21:00:242
2547299129,2cyclictest13820-21wc19:50:262
2547299129,2cyclictest12481-21wc20:55:232
2547299129,2cyclictest11646-21grep19:45:272
2547299128,3cyclictest13582-21sh23:21:442
25472991211,1cyclictest28358-21wc23:30:252
25472991210,2cyclictest32-21ksoftirqd/223:55:482
25472991210,1cyclictest32285-21tr22:35:182
2546999129,2cyclictest8884-21basename20:50:151
2546999129,2cyclictest3698-21ssh21:20:151
2546999129,2cyclictest32730-21taskset22:35:211
2546999129,2cyclictest27715-21sh21:54:181
2546999129,2cyclictest24516-21rm23:48:021
2546999129,2cyclictest24516-21rm23:48:011
2546999129,2cyclictest24094-21ssh21:32:141
2546999129,2cyclictest19300-21cron21:30:011
2546999129,2cyclictest15737-21rm22:45:021
2546999129,2cyclictest14014-21ssh00:00:311
2546999126,5cyclictest38150irq/131-snd_hda21:17:551
2546999123,2cyclictest225-21systemd-journal23:10:241
25469991211,1cyclictest0-21swapper/122:01:141
25469991211,1cyclictest0-21swapper/122:01:131
2546999120,2cyclictest0-21swapper/120:15:131
2546999120,11cyclictest8384-21grep22:20:271
2546999120,11cyclictest30036-21sed21:55:181
2546699129,2cyclictest985-21grep22:35:270
2546699129,2cyclictest29065-21cat19:15:190
2546699129,2cyclictest26343-21rm22:50:280
2546699129,2cyclictest2512-21ssh23:35:130
2546699129,2cyclictest23947-21sed22:30:180
2546699129,2cyclictest22792-21wc22:10:190
2546699129,2cyclictest22755-21grep21:50:300
2546699129,2cyclictest17738-21grep20:00:160
2546699129,2cyclictest12660-21wc23:40:270
2546699129,2cyclictest11466-21ssh21:25:130
2546699129,2cyclictest10269-21cat20:50:260
2546699121,7cyclictest4913-21munin-run19:35:000
2546699121,5cyclictest16968-21/usr/sbin/munin21:05:190
2546699120,11cyclictest4122-21tr20:40:150
2546699120,11cyclictest20883-21cut21:30:200
2546699120,10cyclictest20561-21grep00:05:140
2547499119,2cyclictest30721-21smartctl00:30:173
2547499119,2cyclictest29080-21sort21:35:183
2547499119,2cyclictest28487-21sh23:11:113
2547499119,2cyclictest27734-21chrt19:14:143
2547499119,2cyclictest16450-21taskset22:06:123
2547499119,1cyclictest9817-21wc20:50:203
2547499119,1cyclictest8881-21wc19:40:213
2547499119,1cyclictest6440-21hostname20:45:133
2547499119,1cyclictest3566-21grep19:30:203
2547499119,1cyclictest2788-21taskset21:20:013
2547499119,1cyclictest27696-21cut20:20:193
2547499119,1cyclictest20031-21sh21:10:213
2547499119,1cyclictest12809-21tail20:55:273
2547499119,1cyclictest12601-21uniq00:00:163
2547499119,1cyclictest119662sleep319:48:203
2547499118,2cyclictest802-21cat20:30:283
2547499118,2cyclictest32519-21chrt19:24:213
2547499118,2cyclictest2516-21ssh22:55:403
2547499118,2cyclictest22167-21cut20:10:143
2547499118,2cyclictest22167-21cut20:10:143
2547499118,2cyclictest17086-21cut21:05:193
2547499118,2cyclictest15721-21sed19:55:203
2547499114,3cyclictest20883-21apt-get21:50:163
2547499113,7cyclictest55550irq/126-enp2s0-21:30:173
2547299119,1cyclictest9054-21tr20:50:152
2547299119,1cyclictest28294-21grep20:20:262
2547299119,1cyclictest22544-21sort20:10:162
2547299119,1cyclictest22544-21sort20:10:162
2547299118,2cyclictest5132-21ssh22:00:152
2547299118,2cyclictest5132-21ssh22:00:142
2547299118,2cyclictest32-21ksoftirqd/222:55:132
2547299118,2cyclictest29865-21sort20:25:182
2547299118,2cyclictest2968-21ssh21:20:082
2547299118,2cyclictest2918-21awk20:35:212
2547299118,2cyclictest2445-21grep23:15:232
2547299118,2cyclictest2303-21chrt19:27:082
2547299118,2cyclictest12187-21ssh21:45:022
2547299117,3cyclictest24500-21tr20:15:152
2547299114,6cyclictest28896-21munin-node19:38:392
2547299110,1cyclictest141rcu_preempt21:05:182
2547299110,10cyclictest3459-21ssh23:35:202
2546999119,2cyclictest9684-21ssh23:20:031
2546999119,2cyclictest23106-21ssh00:06:061
2546999119,1cyclictest3517-21grep19:30:191
2546999119,1cyclictest30455-21cat19:20:131
2546999119,1cyclictest29107-21ssh23:30:491
2546999119,1cyclictest28082-21apt-get19:15:131
2546999119,1cyclictest275682sleep119:12:051
2546999119,1cyclictest10025-21basename19:45:151
2546999118,2cyclictest7856-21tr19:40:141
2546999118,2cyclictest730-21cat20:30:271
2546999118,2cyclictest25752-21tr22:50:241
2546999118,2cyclictest1828-21grep20:35:131
2546999118,2cyclictest17458-21gzip21:05:231
2546999118,2cyclictest12796-21expr21:45:141
2546999115,2cyclictest225-21systemd-journal19:29:591
2546999112,8cyclictest869-21diskstats22:55:161
2546999110,1cyclictest0-21swapper/122:26:261
2546699119,1cyclictest19611-21ssh23:45:150
2546699119,1cyclictest19611-21ssh23:45:150
2546699119,1cyclictest1100-21head22:55:170
2546699118,2cyclictest8561-21uname19:40:190
2546699118,2cyclictest7030-21grep20:45:160
2546699118,2cyclictest6231-21cat19:35:210
2546699118,2cyclictest31552-21ssh23:13:330
2546699118,2cyclictest30011-21head21:55:180
2546699118,2cyclictest29937-21fschecks_time20:25:180
2546699118,2cyclictest29657-21ssh00:30:050
2546699118,2cyclictest23295-21sed00:25:240
2546699118,2cyclictest18978-21rm23:25:160
2546699118,2cyclictest18429-21tail23:05:220
2546699118,2cyclictest14791-21cat00:20:240
2546699118,2cyclictest14561-21ssh22:25:050
2546699118,2cyclictest13260-21cron20:59:590
2546699118,2cyclictest11123-21ssh23:59:490
2546699116,4cyclictest7136-21apache222:06:230
2546699115,3cyclictest0-21swapper/021:45:130
2546699114,6cyclictest7135-21apache223:03:390
2546699114,1cyclictest7378-21apt-get22:40:040
2546699113,1cyclictest18268-21diskmemload21:16:050
2546699111,8cyclictest0-21swapper/019:25:130
2546699110,10cyclictest29460-21sed23:50:250
2547499108,2cyclictest18683-21grep20:00:253
2547499108,1cyclictest4608-21cut20:40:163
2547499108,1cyclictest33852sleep320:36:093
2547499107,2cyclictest31205-21ssh21:55:283
2547299108,1cyclictest274792sleep219:10:582
2547299106,3cyclictest31237-21sed19:20:182
2546999109,1cyclictest71862sleep119:38:091
2546999109,1cyclictest27715-21cat20:20:201
2546999109,1cyclictest11487-21cat20:55:151
2546999108,1cyclictest7837-21sort20:45:241
2546999108,1cyclictest4415-21expr20:40:161
2546999108,1cyclictest19552-21wget20:05:141
2546999108,1cyclictest13755-21cat21:00:111
2546999107,2cyclictest15147-21cat19:55:161
2546999101,1cyclictest12853-21df19:50:151
2546699108,1cyclictest34942sleep020:37:300
2546699108,1cyclictest31565-21tr19:20:210
2546699108,1cyclictest28599-21ssh00:10:030
2546699108,1cyclictest26981-21cat20:20:150
2546699107,2cyclictest30702-21ssh21:35:530
2546699102,2cyclictest0-21swapper/022:00:150
2546699102,2cyclictest0-21swapper/022:00:140
2546699100,6cyclictest11664-21grep19:45:270
2546699100,2cyclictest10414-21sshd22:22:020
254749996,2cyclictest29187-21cat20:25:023
254669998,1cyclictest27056-21mailstats19:10:250
254669998,1cyclictest12790-21expr19:50:150
254669997,2cyclictest19998-21sh20:05:150
254669997,1cyclictest13600-21cut21:00:110
254669996,2cyclictest21890-21apache_accesses20:10:130
254669996,2cyclictest21890-21apache_accesses20:10:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional