You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 08:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Jul 15, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2846329381,8sleep00-21swapper/019:06:450
2854528774,9sleep30-21swapper/319:07:503
2842828767,9sleep20-21swapper/219:06:202
207582830,0sleep10-21swapper/123:05:001
2658528272,6sleep10-21swapper/119:05:101
13512640,0sleep00-21swapper/021:16:120
28804993029,1cyclictest39-21ksoftirqd/300:21:173
2880299280,28cyclictest0-21swapper/221:32:012
2880299280,28cyclictest0-21swapper/221:32:012
2880299270,26cyclictest0-21swapper/223:55:132
2880299240,24cyclictest0-21swapper/200:33:122
2880299240,24cyclictest0-21swapper/200:00:302
2879399240,0cyclictest0-21swapper/000:25:060
28804992320,2cyclictest28720-21chrt23:09:433
2880499230,21cyclictest0-21swapper/321:38:503
2880299230,3cyclictest0-21swapper/223:08:472
2880299230,23cyclictest0-21swapper/223:27:282
2879899230,2cyclictest0-21swapper/122:33:441
2879899230,21cyclictest0-21swapper/122:53:451
2880499220,1cyclictest141rcu_preempt21:27:493
2879399220,18cyclictest0-21swapper/000:04:090
2879399220,0cyclictest0-21swapper/022:22:300
28804992118,2cyclictest18973-21chrt23:41:583
2880499210,20cyclictest14068-21kworker/u8:222:43:393
2879399211,8cyclictest0-21swapper/021:44:550
28793992118,2cyclictest24200-21chrt00:05:020
2879399210,4cyclictest0-21swapper/021:25:130
2880499201,16cyclictest0-21swapper/323:50:583
2880299200,20cyclictest0-21swapper/223:22:002
28798992018,1cyclictest27979-21chrt22:10:301
2879399200,18cyclictest0-21swapper/000:15:540
28804991916,2cyclictest6182-21chrt00:32:223
2880299190,18cyclictest0-21swapper/221:47:332
2879399190,3cyclictest0-21swapper/023:20:550
2880499189,8cyclictest0-21swapper/322:10:153
28804991816,1cyclictest11345-21chrt23:37:323
2880499180,4cyclictest0-21swapper/321:14:443
2880499180,17cyclictest0-21swapper/323:10:223
2880299185,7cyclictest21853-21diskmemload22:10:002
28798991816,2cyclictest0-21swapper/100:18:251
28798991810,3cyclictest25232-21idleruntime-cro20:10:001
2879899180,17cyclictest0-21swapper/121:40:551
28804991715,2cyclictest39-21ksoftirqd/321:49:433
28804991713,4cyclictest11708-21kworker/u8:019:10:043
2880499170,3cyclictest39-21ksoftirqd/322:58:003
2880499170,3cyclictest39-21ksoftirqd/322:57:593
2880499170,16cyclictest0-21swapper/322:50:353
2880299170,4cyclictest0-21swapper/223:15:182
2880299170,1cyclictest141rcu_preempt21:10:362
2880299170,17cyclictest0-21swapper/200:19:282
2879899170,16cyclictest0-21swapper/121:37:591
2879399175,6cyclictest11043-21awk22:40:000
28793991714,2cyclictest27650-21chrt23:28:030
2879399170,2cyclictest0-21swapper/022:49:210
2879399170,1cyclictest0-21swapper/000:14:580
2880499169,6cyclictest21853-21diskmemload23:20:193
2880499163,2cyclictest0-21swapper/321:30:133
2880499163,2cyclictest0-21swapper/321:30:133
2880499162,14cyclictest0-21swapper/300:01:303
2880499161,11cyclictest39-21ksoftirqd/323:30:593
2880499160,16cyclictest0-21swapper/320:10:143
2880299160,16cyclictest0-21swapper/222:27:092
2880299160,16cyclictest0-21swapper/221:37:372
2880299160,16cyclictest0-21swapper/200:28:412
2880299160,0cyclictest0-21swapper/223:14:432
28798991613,2cyclictest29794-21chrt00:27:041
28798991611,3cyclictest38150irq/131-snd_hda19:40:011
28798991611,3cyclictest38150irq/131-snd_hda19:40:001
2879899160,15cyclictest0-21swapper/100:31:181
80872150,0chrt0-21swapper/222:56:402
80872150,0chrt0-21swapper/222:56:392
2880499158,4cyclictest39-21ksoftirqd/321:17:413
2880499157,6cyclictest94-21kswapd000:35:143
28804991513,2cyclictest39-21ksoftirqd/321:00:193
28804991511,3cyclictest9692-21grep00:15:183
2880499150,14cyclictest0-21swapper/300:13:123
2880299153,1cyclictest141rcu_preempt00:07:422
28802991514,1cyclictest0-21swapper/219:55:582
2880299150,13cyclictest0-21swapper/200:35:242
28798991513,2cyclictest10550irq/122-ahci[0023:44:591
2879899150,3cyclictest0-21swapper/121:53:441
255882150,0chrt0-21swapper/222:48:182
2880499141,3cyclictest4226-21ssh21:56:333
28804991412,2cyclictest39-21ksoftirqd/322:06:163
28804991412,1cyclictest6202-21apt-get19:30:133
28804991412,1cyclictest22328-21apt-get23:25:143
2880499141,0cyclictest0-21swapper/323:49:293
2880499140,3cyclictest141rcu_preempt22:39:183
2880499140,1cyclictest0-21swapper/323:00:373
2880499140,14cyclictest0-21swapper/322:25:573
2880299140,14cyclictest0-21swapper/222:20:462
2879899140,14cyclictest0-21swapper/122:09:331
2879899140,10cyclictest0-21swapper/122:47:131
28793991410,3cyclictest8877-21cat22:00:000
2880499139,3cyclictest39-21ksoftirqd/321:21:243
2880499139,3cyclictest24755-21grep00:05:153
2880499137,6cyclictest21853-21diskmemload00:29:473
2880499132,10cyclictest30683-21cpuspeed_turbos20:20:163
28804991312,1cyclictest39-21ksoftirqd/322:01:233
28804991311,2cyclictest0-21swapper/322:19:343
28804991310,2cyclictest24656-21wc20:05:273
28804991310,2cyclictest18887-21df19:55:173
28804991310,2cyclictest1691-21tr19:20:163
28804991310,2cyclictest15598-21awk00:00:003
28804991310,2cyclictest11647-21grep22:20:223
2880499130,2cyclictest141rcu_preempt20:30:123
2880299139,3cyclictest31537-21grep23:30:192
2880299139,2cyclictest7916-21tr20:40:152
2880299138,2cyclictest0-21swapper/221:25:432
2880299134,6cyclictest0-21swapper/222:10:182
28802991310,2cyclictest8851-21tr19:35:142
28802991310,2cyclictest8851-21tr19:35:142
28802991310,2cyclictest5253-21sshd21:57:212
28802991310,2cyclictest27657-21cat20:15:002
28802991310,2cyclictest24375-21wc20:05:232
28802991310,2cyclictest13299-21grep20:50:182
28802991310,2cyclictest11551-21ssh22:40:132
2880299130,2cyclictest0-21swapper/222:04:372
2880299130,13cyclictest0-21swapper/223:45:302
2880299130,0cyclictest0-21swapper/222:17:322
2879899139,3cyclictest5945-21dump-pmu-power19:30:001
2879899139,3cyclictest23237-21grep21:10:171
2879899139,3cyclictest10823-21ssh00:35:191
2879899138,4cyclictest6071-21ssh23:54:051
2879899138,4cyclictest22481-21ssh00:03:511
2879899138,2cyclictest15091-21ssh23:59:371
2879899137,5cyclictest7723-21cut23:35:191
28798991310,3cyclictest21853-21diskmemload23:19:301
28798991310,2cyclictest5080-21cat20:35:001
28798991310,2cyclictest4806-21tr22:55:161
28798991310,2cyclictest4806-21tr22:55:151
28798991310,2cyclictest3292-21grep22:15:241
28798991310,2cyclictest30134-21grep19:10:211
28798991310,2cyclictest17004-21cron22:05:001
28798991310,2cyclictest11881-21grep19:40:161
2879899130,2cyclictest14817-21grep23:20:191
2879899130,13cyclictest0-21swapper/123:49:211
2879399139,3cyclictest551-21ssh21:35:180
2879399136,6cyclictest24706-21ntp_states23:45:210
2879399131,7cyclictest17017-21idleruntime-cro22:05:000
2879399131,6cyclictest12378-21readlink23:00:000
2879399131,6cyclictest12378-21readlink23:00:000
2879399131,3cyclictest225-21systemd-journal20:10:000
28793991312,1cyclictest13-21ksoftirqd/023:57:460
28793991310,2cyclictest29768-21grep19:10:190
28793991310,2cyclictest2590-21cron20:30:000
28793991310,2cyclictest25342-21grep00:25:000
28793991310,2cyclictest11107-21grep20:45:200
2879399130,7cyclictest4008-21perf22:55:000
2880499129,2cyclictest5730-21cut23:15:153
2880499129,2cyclictest27425-21grep22:30:143
2880499129,2cyclictest22340-21grep20:00:273
2880499129,2cyclictest17097-21systemctl21:00:003
2880499129,2cyclictest10029-21basename20:45:143
2880499124,7cyclictest39-21ksoftirqd/321:40:133
2880499122,6cyclictest39-21ksoftirqd/322:50:003
28804991210,1cyclictest0-21swapper/321:05:133
2880299129,2cyclictest8052-21grep23:35:222
2880299129,2cyclictest5988-21sed20:35:192
2880299129,2cyclictest4917-21grep19:25:192
2880299129,2cyclictest16944-21grep23:40:242
2880299129,2cyclictest15162-21sed19:45:272
2880299128,3cyclictest29105-21sed22:30:272
2880299128,3cyclictest2787-21wc20:30:002
2880299122,3cyclictest225-21systemd-journal20:30:202
28802991210,1cyclictest8415-21cat19:35:002
2880299120,0cyclictest0-21swapper/223:00:012
2879899129,2cyclictest8737-21sed21:20:271
2879899129,2cyclictest32135-21tail20:20:281
2879899129,2cyclictest18118-21cat22:25:001
2879899129,2cyclictest14692-21cron20:55:011
2879899129,2cyclictest11269-21wc20:45:221
2879899128,3cyclictest3215-21taskset23:32:431
2879899128,3cyclictest32130-21cut21:15:231
2879899128,2cyclictest17431-21grep21:00:161
2879899128,2cyclictest17036-21grep21:25:271
2879899127,4cyclictest28935-21ssh00:07:171
2879899127,4cyclictest17760-21ssh22:43:441
2879899122,6cyclictest225-21systemd-journal20:15:011
28798991211,1cyclictest0-21swapper/123:06:011
2879399129,3cyclictest0-21swapper/021:51:100
2879399129,2cyclictest6673-21chrt23:35:120
2879399129,2cyclictest30405-21cut23:10:180
2879399129,2cyclictest29495-21rm21:33:270
2879399129,2cyclictest29495-21rm21:33:260
2879399129,2cyclictest28835-21grep20:15:170
2879399129,2cyclictest2543-21grep19:20:210
2879399129,2cyclictest22837-21cron20:05:000
2879399129,2cyclictest20458-21grep21:05:170
2879399129,2cyclictest1979-21grep00:30:140
2879399129,2cyclictest18987-21grep21:00:280
2879399129,2cyclictest18085-21sh19:55:000
2879399129,2cyclictest12332-21grep22:40:170
2879399128,3cyclictest4053-21grep19:25:140
2879399128,3cyclictest2621-21uniq22:15:180
2879399127,3cyclictest21986-21ssh23:05:180
2879399125,5cyclictest4967-21cron20:34:590
2879399121,10cyclictest0-21swapper/022:26:440
2879399121,0cyclictest0-21swapper/021:13:270
2879399120,1cyclictest0-21swapper/000:39:500
2880499118,2cyclictest8744-21uname20:40:203
2880499118,2cyclictest28156-21cut20:15:143
2880499118,2cyclictest26618-21grep21:50:263
2880499118,2cyclictest14287-21head19:45:173
2880499118,2cyclictest11724-21df19:40:153
2880499118,2cyclictest10725-21sendmail-msp19:40:003
2880499118,2cyclictest10725-21sendmail-msp19:40:003
2880499115,6cyclictest32425-21kworker/3:019:28:533
2880499110,10cyclictest14475-21taskset20:52:153
2880299118,2cyclictest5378-21ssh21:19:202
2880299118,2cyclictest4714-21ssh23:53:012
2880299118,2cyclictest32106-21cut19:15:172
2880299118,2cyclictest31853-21basename20:20:242
2880299118,2cyclictest30565-21ssh22:50:292
2880299118,2cyclictest3016-21ssh00:10:462
2880299118,2cyclictest29729-21sed20:15:262
2880299118,2cyclictest20843-21uname21:05:192
2880299118,2cyclictest17695-21taskset00:20:152
2880299118,2cyclictest15639-21cut20:55:182
2880299117,3cyclictest25127-21grep21:50:162
2880299116,3cyclictest4409-21grep22:35:192
2880299111,5cyclictest225-21systemd-journal19:45:012
28802991111,0cyclictest0-21swapper/221:40:372
2880299110,9cyclictest0-21swapper/221:20:132
2879899119,1cyclictest16682-21sed20:55:281
2879899118,2cyclictest8428-21cat19:35:001
2879899118,2cyclictest7730-21ssh21:59:171
2879899118,2cyclictest6148-21ssh00:13:131
2879899118,2cyclictest4334-21ssh22:35:181
2879899118,2cyclictest29301-21grep20:15:221
2879899118,2cyclictest28858-21ssh21:32:581
2879899118,2cyclictest28858-21ssh21:32:571
2879899118,2cyclictest22449-21wget23:25:141
2879899118,2cyclictest21851-21basename21:10:001
2879899118,2cyclictest18874-21tr22:25:141
2879899118,2cyclictest1413-21uname20:25:191
2879899117,1cyclictest13475-21kworker/1:121:45:111
2879899115,4cyclictest0-21swapper/100:23:521
2879899113,6cyclictest18400-21basename19:55:161
2879899112,7cyclictest7362-21munin-run20:40:001
2879899112,6cyclictest7637-21apt-get20:40:141
2879399118,2cyclictest9402-21cat19:35:190
2879399118,2cyclictest9402-21cat19:35:180
2879399118,2cyclictest5929-21cut20:35:190
2879399118,2cyclictest28837-21cat22:30:260
2879399118,2cyclictest27277-21ssh22:10:230
2879399118,2cyclictest2271-21ssh23:51:060
2879399118,2cyclictest19459-21ssh23:42:190
2879399118,2cyclictest19344-21ssh21:46:050
2879399118,2cyclictest19087-21ssh23:03:430
2879399118,2cyclictest17602-21cut22:05:160
2879399118,2cyclictest12415-21grep19:40:200
2879399118,2cyclictest12357-21rm21:23:160
2879399117,3cyclictest31539-21cut23:30:180
2880499108,1cyclictest8302sleep319:16:023
2880499108,1cyclictest5610-21ls20:35:163
2880499100,10cyclictest0-21swapper/320:25:143
2880299108,1cyclictest17005-21cat19:50:202
2880299108,1cyclictest10551-21sed20:45:162
2880299107,2cyclictest30830-21taskset19:11:092
2880299107,2cyclictest1993-21df_abs19:20:172
2880299106,3cyclictest23031-21cat20:05:002
2880299100,5cyclictest225-21systemd-journal21:05:012
2879899108,1cyclictest32470-21wc19:15:201
2879899108,1cyclictest21865-21grep20:00:211
2879899107,2cyclictest29970-21tr23:10:161
2879899107,2cyclictest18066-21cron19:55:001
2879899107,2cyclictest1430-21cat19:20:131
2879899107,2cyclictest13915-21cat19:45:151
2879399108,1cyclictest6634-21cat19:30:130
2879399108,1cyclictest32437-21memory19:15:200
2879399108,1cyclictest14758-21cut19:45:230
2879399107,2cyclictest5926-21cut23:15:140
2879399107,2cyclictest31216-21head20:20:170
2879399107,2cyclictest20456-21sendmail20:00:010
2879399107,2cyclictest14399-21taskset20:51:170
2879399102,4cyclictest225-21systemd-journal21:00:000
2879399100,5cyclictest225-21systemd-journal20:15:010
288049991,2cyclictest39-21ksoftirqd/319:50:173
287939996,2cyclictest9255-21grep20:40:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional