You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 22:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Dec 21, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
488129566,25sleep00-21swapper/007:07:530
474529371,18sleep30-21swapper/307:06:063
488828674,8sleep20-21swapper/207:07:582
308228663,21sleep10-21swapper/107:05:171
514099250,2cyclictest0-21swapper/109:00:211
5140992222,0cyclictest0-21swapper/108:45:141
5140992219,1cyclictest0-21swapper/111:20:001
514799210,0cyclictest0-21swapper/309:55:583
5147991918,1cyclictest0-21swapper/308:55:143
514799170,2cyclictest0-21swapper/309:30:143
5145991614,1cyclictest0-21swapper/212:20:142
5137991615,1cyclictest0-21swapper/011:35:150
5140991513,1cyclictest0-21swapper/112:05:181
514099151,12cyclictest14927-21grep07:30:161
514799149,3cyclictest24670-21tr07:50:163
514599148,4cyclictest455-21dbus-daemon08:30:232
5145991411,2cyclictest25147-21wc10:05:212
5145991410,3cyclictest13544-21cat07:25:212
514099144,6cyclictest225-21systemd-journal09:09:541
5140991410,3cyclictest27524-21head12:25:171
5140991410,3cyclictest14710-21grep09:45:151
513799149,3cyclictest25384-21wc09:00:010
513799145,6cyclictest30496-21cut11:25:140
5137991411,2cyclictest15777-21sh08:40:000
5137991410,3cyclictest19584-21tr12:10:150
5137991410,3cyclictest15493-21sort07:30:190
5137991410,2cyclictest24154-21cut12:20:140
514799134,6cyclictest0-21swapper/311:45:263
5147991310,2cyclictest6750-21grep09:25:253
5147991310,2cyclictest6658-21cat11:40:273
5147991310,2cyclictest6357-21wc07:10:203
5147991310,2cyclictest2545-21cat08:10:183
5147991310,2cyclictest20951-21grep08:50:143
5147991310,2cyclictest20090-21sed08:45:263
5147991310,2cyclictest19694-21grep07:40:153
5147991310,2cyclictest14110-21seq08:35:143
514799130,12cyclictest19072-21cron12:10:003
514599139,3cyclictest6131-21apt-config10:35:002
514599135,4cyclictest225-21systemd-journal10:20:012
514599135,4cyclictest225-21systemd-journal10:20:002
5145991310,2cyclictest9605-21sh09:34:592
5145991310,2cyclictest3627-21cron10:30:012
5145991310,2cyclictest27749-21dump-pmu-power09:04:592
514099139,3cyclictest12843-21ls08:30:271
514099139,3cyclictest12043-21cat11:55:001
514099134,3cyclictest225-21systemd-journal10:43:101
5140991310,2cyclictest8420-21wc11:45:191
5140991310,2cyclictest4072-21grep08:15:141
5140991310,2cyclictest31457-21grep11:25:201
5140991310,2cyclictest2525-21wc11:35:001
5140991310,2cyclictest23375-21grep12:15:261
5140991310,2cyclictest23375-21grep12:15:261
5140991310,2cyclictest23291-21wc07:45:231
513799139,3cyclictest29320-21ps09:05:250
513799139,2cyclictest6240-21wc08:20:000
513799138,3cyclictest19729-21grep07:40:150
513799132,6cyclictest13383-21munin-run10:50:000
5137991310,2cyclictest5621-21grep10:30:270
5137991310,2cyclictest27309-21grep10:10:180
5137991310,2cyclictest19267-21wc09:55:010
513799130,8cyclictest0-21swapper/009:45:140
513799130,11cyclictest2981-21grep09:20:140
514799129,2cyclictest4205-21tr08:15:153
514799129,2cyclictest32000-21grep09:10:263
514799129,2cyclictest31446-21grep08:05:003
514799129,2cyclictest30822-21grep12:30:283
514799129,2cyclictest27774-21cat09:04:593
514799129,2cyclictest24180-21cat11:10:183
514799129,2cyclictest18706-21grep09:50:273
514799129,2cyclictest1854-21cat11:30:293
514799129,2cyclictest15367-21head09:45:193
514799129,2cyclictest12989-21grep08:30:273
514799128,3cyclictest32271-21grep12:35:153
514799128,3cyclictest15535-21grep07:30:183
514799128,2cyclictest11372-21cat11:50:253
514599129,2cyclictest9144-21sed11:45:262
514599129,2cyclictest9114-21tr10:40:152
514599129,2cyclictest7684-21tr08:20:222
514599129,2cyclictest7684-21tr08:20:212
514599129,2cyclictest32361-21head10:20:182
514599129,2cyclictest28723-21grep09:05:212
514599129,2cyclictest28696-21apt10:15:002
514599129,2cyclictest27030-21tr07:55:152
514599129,2cyclictest23897-21turbostat10:05:012
514599129,2cyclictest23847-21perf12:20:012
514599129,2cyclictest23847-21perf12:20:002
514599129,2cyclictest2039-21head09:15:272
514599129,2cyclictest17968-21grep07:35:172
514599129,2cyclictest14407-21sed10:50:192
514599129,2cyclictest10903-21grep07:20:162
514599128,2cyclictest15770-21cron08:40:002
514599128,2cyclictest14439-21wc12:00:012
514599124,6cyclictest22990-21cat07:45:212
514599120,9cyclictest0-21swapper/212:00:112
514099129,2cyclictest8666-21apt08:25:011
514099129,2cyclictest8666-21apt08:25:001
514099129,2cyclictest4999-21wc11:40:001
514099129,2cyclictest32203-21grep08:05:171
514099129,2cyclictest29476-21tr08:00:131
514099129,2cyclictest28706-21head11:20:181
514099129,2cyclictest25876-21head10:05:301
514099129,2cyclictest24510-21wc08:55:241
514099129,2cyclictest24291-21tr11:10:191
514099129,2cyclictest17528-21grep07:35:151
514099129,2cyclictest16779-21grep08:40:181
514099129,2cyclictest16276-21grep10:55:131
514099127,4cyclictest5804-21taskset10:32:241
514099124,4cyclictest225-21systemd-journal09:34:591
513799129,2cyclictest8253-21head09:30:180
513799129,2cyclictest7284-21grep08:20:180
513799129,2cyclictest7284-21grep08:20:180
513799129,2cyclictest713-21wc10:20:270
513799129,2cyclictest27687-21cron11:19:590
513799129,2cyclictest17106-21grep10:55:190
513799129,2cyclictest11042-21perf08:30:000
513799128,3cyclictest28932-21cron08:00:000
513799128,3cyclictest23730-21sed07:45:290
513799128,3cyclictest14383-21apt09:45:000
513799126,2cyclictest225-21systemd-journal07:20:000
513799121,6cyclictest28887-21awk12:30:000
514799118,2cyclictest7931-21grep07:15:123
514799118,2cyclictest27960-21basename11:20:113
514799118,2cyclictest27312-21head10:10:193
514799118,2cyclictest25494-21taskset11:15:103
514799118,2cyclictest24133-21sh07:49:593
514799118,2cyclictest21485-21grep11:05:183
514799118,2cyclictest20604-21ps12:10:223
514799118,2cyclictest15201-21cut12:00:183
514799118,2cyclictest14403-21head10:50:193
514799118,2cyclictest13398-21grep10:50:003
514799118,2cyclictest13390-21cat07:25:193
514799118,2cyclictest11447-21gzip07:20:213
514799115,5cyclictest16779-21head10:55:163
514799110,10cyclictest0-21swapper/310:15:163
514799110,10cyclictest0-21swapper/310:15:153
514599118,2cyclictest9857-21sed08:25:172
514599118,2cyclictest6833-21grep09:25:262
514599118,2cyclictest6695-21grep11:40:272
514599118,2cyclictest3618-21seq08:14:312
514599118,2cyclictest32296-21chrt11:28:022
514599118,2cyclictest27442-21taskset11:16:582
514599118,2cyclictest22901-21cat08:54:552
514599118,2cyclictest20903-21aten2_r7power_v11:05:152
514599118,2cyclictest20672-21expr12:10:232
514599118,2cyclictest20432-21taskset08:48:452
514599118,2cyclictest19653-21expr09:55:182
514599118,2cyclictest18974-21taskset12:08:482
514599118,2cyclictest17761-21sed09:50:182
514599118,2cyclictest16716-21cut08:40:172
514599118,2cyclictest15663-21missed_timers09:45:202
514599118,2cyclictest1452-21grep12:40:002
514599118,2cyclictest13916-21tail09:40:252
514599118,2cyclictest11480-21grep09:35:272
514599115,2cyclictest225-21systemd-journal07:15:002
514099118,2cyclictest904-21taskset10:20:421
514099118,2cyclictest32282-21taskset09:12:221
514099118,2cyclictest30366-21tr12:30:241
514099118,2cyclictest30075-21cut10:15:231
514099118,2cyclictest30075-21cut10:15:221
514099118,2cyclictest26048-21taskset12:22:541
514099118,2cyclictest21469-21awk12:15:011
514099118,2cyclictest21267-21df11:05:171
514099118,2cyclictest20569-21date11:04:591
514099118,2cyclictest2012-21sed09:15:271
514099118,2cyclictest19123-21ls09:55:001
514099118,2cyclictest13042-21chrt10:45:571
514099118,2cyclictest11572-21grep09:35:271
514099118,2cyclictest10706-21chrt08:25:581
514099118,2cyclictest10371-21tr07:20:131
514099117,3cyclictest28763-21cat10:15:001
514099117,3cyclictest2280-21head08:10:171
514099114,3cyclictest225-21systemd-journal07:25:171
514099113,4cyclictest225-21systemd-journal09:59:591
513799118,2cyclictest8406-21cut11:45:190
513799118,2cyclictest7429-21uname10:35:170
513799118,2cyclictest6989-21grep07:10:260
513799118,2cyclictest6323-21grep11:40:220
513799118,2cyclictest5362-21tr09:25:150
513799118,2cyclictest31112-21grep09:10:170
513799118,2cyclictest29241-21grep12:30:150
513799118,2cyclictest28241-21cut11:20:150
513799118,2cyclictest21219-21expr11:05:160
513799118,2cyclictest18077-21grep12:05:200
513799118,2cyclictest17535-21tr07:35:150
513799118,2cyclictest16328-21expr08:40:140
513799118,2cyclictest15379-21cut12:00:190
513799118,2cyclictest14437-21cat12:00:000
513799118,2cyclictest14044-21ls10:50:170
513799118,2cyclictest13006-21df07:25:170
513799118,2cyclictest10852-21taskset10:43:200
513799117,3cyclictest26693-21cut09:00:210
513799117,3cyclictest2409-21cron11:35:000
513799112,6cyclictest32253-21grep08:05:160
514799108,1cyclictest9717-21which08:25:163
514799108,1cyclictest6769-21expr10:35:143
514799108,1cyclictest642-21cat09:15:163
514799108,1cyclictest4303-21expr10:30:163
514799108,1cyclictest32597-21memory10:20:213
514799108,1cyclictest2907-21cut10:25:233
514799108,1cyclictest27773-21cat12:25:193
514799108,1cyclictest24577-21expr10:05:163
514799108,1cyclictest23354-21wc10:00:283
514799108,1cyclictest10199-21gzip10:40:233
514799107,2cyclictest8048-21cat08:20:283
514799107,2cyclictest8048-21cat08:20:273
514799107,2cyclictest31794-21grep08:05:153
514799107,2cyclictest29004-21cat09:05:233
514799107,2cyclictest2792-21cut09:20:153
514799107,2cyclictest27086-21cstates07:55:153
514799107,2cyclictest22389-21cut12:15:173
514799107,2cyclictest22389-21cut12:15:163
514799107,2cyclictest19948-21sort11:00:263
514799107,2cyclictest16829-21sed08:40:183
514799107,2cyclictest14206-21taskset09:43:023
514799106,3cyclictest24155-21tr12:20:133
514799103,5cyclictest9877-21apt-get09:35:143
514799102,4cyclictest225-21systemd-journal11:35:053
514599108,1cyclictest8675-21uname07:15:172
514599108,1cyclictest4661-21head08:15:172
514599108,1cyclictest3730-21wc09:20:202
514599108,1cyclictest31995-21cat08:05:162
514599108,1cyclictest29094-21chrt12:30:052
514599108,1cyclictest28564-21taskset12:26:102
514599108,1cyclictest27984-21apache_processe11:20:132
514599108,1cyclictest26133-21seq07:50:282
514599108,1cyclictest24208-21chrt08:55:212
514599108,1cyclictest23148-21apache_accesses11:10:102
514599108,1cyclictest11635-21taskset11:51:132
514599107,2cyclictest4006-21grep11:35:232
514599107,2cyclictest30249-21chrt08:00:182
514599107,2cyclictest2126-21taskset11:31:242
514599107,2cyclictest19797-21grep07:40:152
514599107,2cyclictest19157-21cut11:00:172
514599102,2cyclictest141rcu_preempt10:50:002
514599100,9cyclictest17882-21seq10:57:102
514099108,1cyclictest6138-21grep07:10:181
514099108,1cyclictest2713-21taskset09:20:111
514099108,1cyclictest25976-21ls07:50:261
514099108,1cyclictest142492sleep109:43:351
514099108,1cyclictest13124-21uname11:55:201
514099107,2cyclictest9104-21cat07:15:211
514099107,2cyclictest7232-21grep10:35:161
514099107,2cyclictest3304-21taskset10:25:541
514099107,2cyclictest21576-21sed08:50:171
514099107,2cyclictest15592-21chrt10:53:161
514099107,2cyclictest15579-21chrt08:37:361
514099107,2cyclictest15277-21cut12:00:191
514099107,2cyclictest1053-21taskset12:35:401
514099106,3cyclictest7066-21taskset11:43:301
514099101,5cyclictest225-21systemd-journal10:04:441
514099100,9cyclictest27709-21if_enp2s007:55:181
514099100,9cyclictest21681-21chrt07:44:081
513799108,2cyclictest11490-21mailstats07:20:210
513799108,1cyclictest32450-21expr12:35:180
513799108,1cyclictest264532sleep007:54:230
513799108,1cyclictest238452sleep010:04:370
513799108,1cyclictest22888-21chrt08:54:460
513799108,1cyclictest2140-21taskset09:16:080
513799108,1cyclictest1543-21basename10:25:150
513799108,1cyclictest13057-21taskset08:30:420
513799107,2cyclictest30937-21grep08:00:250
513799107,2cyclictest24159-21taskset10:05:150
513799107,2cyclictest22491-21sed12:15:180
513799107,2cyclictest22491-21sed12:15:170
513799107,2cyclictest20991-21grep09:55:290
513799107,2cyclictest18772-21expr08:45:160
513799107,2cyclictest11459-21cat09:35:260
513799106,3cyclictest24816-21sed11:10:240
513799104,2cyclictest225-21systemd-journal11:54:590
513799102,4cyclictest225-21systemd-journal10:59:590
513799100,9cyclictest29449-21df10:15:180
513799100,9cyclictest29449-21df10:15:180
51479996,2cyclictest17510-21grep07:35:163
51479996,2cyclictest12324-21cut11:55:163
51479990,6cyclictest30322-21sshd11:25:093
51459998,1cyclictest31541-21ntpq09:10:212
51459997,1cyclictest15679-21expr07:30:182
51459996,2cyclictest7225-21grep10:35:162
51409995,2cyclictest5196-21grep09:25:141
51379996,2cyclictest2874-21sendmail_mailqu08:10:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional