You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 16:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Jun 25, 2026 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21692181103,27sleep20-21swapper/207:08:072
21302165108,28sleep30-21swapper/307:07:373
21432163105,29sleep10-21swapper/107:07:471
20132160102,28sleep00-21swapper/007:06:040
250692640,1sleep20-21swapper/209:10:572
229482620,2sleep20-21swapper/211:08:512
157852610,2sleep20-21swapper/212:15:172
321292590,3chrt0-21swapper/012:27:320
226052590,8sleep228-21ksoftirqd/211:31:392
175752570,2sleep30-21swapper/310:40:393
126162540,1sleep20-21swapper/209:25:252
312282530,11sleep122-21ksoftirqd/110:50:241
166472530,1sleep20-21swapper/209:52:512
109972530,4sleep0131rcuc/009:25:120
299052520,1sleep30-21swapper/312:25:263
130252520,2sleep30-21swapper/308:45:233
130252520,2sleep30-21swapper/308:45:223
175022510,3sleep334-21ksoftirqd/311:28:313
169592510,2sleep10-21swapper/109:29:421
65092500,1sleep10-21swapper/110:09:431
10262480,1sleep20-21swapper/209:17:192
245662450,2chrt0-21swapper/112:21:311
32362420,2sleep00-21swapper/009:43:010
183602410,1sleep10-21swapper/111:29:211
258899399,14cyclictest0-21swapper/010:03:250
2588993912,10cyclictest6338-21ssh09:20:480
258899388,14cyclictest0-21swapper/012:22:170
2588993812,9cyclictest22370-21diskmemload12:30:250
2588993811,10cyclictest22370-21diskmemload09:18:330
2588993810,16cyclictest0-21swapper/010:30:040
2588993810,15cyclictest0-21swapper/010:15:440
2588993810,15cyclictest0-21swapper/010:15:430
2588993810,12cyclictest13086-21ssh11:00:460
258899377,15cyclictest0-21swapper/009:59:020
2588993731,4cyclictest0-21swapper/010:51:580
2588993711,10cyclictest22370-21diskmemload10:11:410
258899369,8cyclictest18111-21ssh11:29:070
258899369,16cyclictest0-21swapper/012:00:460
258899369,10cyclictest6215-21ssh10:09:260
2588993611,9cyclictest22370-21diskmemload10:29:200
130772360,2sleep20-21swapper/210:14:302
258899357,8cyclictest22234-21ssh10:20:240
258899357,12cyclictest23274-21runrttasks11:55:450
2588993513,8cyclictest19393-21sh10:42:280
2588993511,10cyclictest8837-21/usr/sbin/munin11:45:220
2588993510,10cyclictest22228-21ssh11:31:150
258899349,9cyclictest22370-21diskmemload09:37:030
258899349,8cyclictest12503-21ssh12:12:480
258899348,9cyclictest19236-21sed09:30:260
258899348,9cyclictest19236-21sed09:30:250
258899348,8cyclictest0-21swapper/011:41:470
258899348,10cyclictest22370-21diskmemload11:21:300
258899347,12cyclictest19721-21rm11:53:580
258899347,12cyclictest19721-21rm11:53:570
258899346,9cyclictest0-21swapper/012:19:470
187102340,3sleep10-21swapper/111:05:181
87672330,2sleep20-21swapper/210:35:082
259199336,25cyclictest34-21ksoftirqd/309:11:393
258899339,8cyclictest0-21swapper/010:46:540
258899338,10cyclictest9832-21memory07:25:170
258899336,11cyclictest17653-21ssh11:05:060
2588993328,4cyclictest0-21swapper/009:00:280
259099321,7cyclictest21746-21memory10:20:192
2589993226,4cyclictest53050irq/142-enp3s0-10:28:101
2589993225,5cyclictest640-21snmpd11:15:581
258899328,9cyclictest0-21swapper/008:05:190
258899328,8cyclictest22370-21diskmemload12:35:510
258899328,8cyclictest22370-21diskmemload12:35:500
258899328,10cyclictest0-21swapper/009:50:100
258899327,15cyclictest0-21swapper/012:06:020
258899327,15cyclictest0-21swapper/009:14:130
258899327,14cyclictest0-21swapper/011:16:370
258899326,10cyclictest11512-21ssh10:36:210
258899321,6cyclictest16359-21cstates08:55:140
2588993210,6cyclictest28126-21users11:35:260
2590993123,6cyclictest3449-21df08:25:152
259099311,27cyclictest3128-21ssh10:06:212
258899319,8cyclictest7942-21df_inode08:35:130
258899319,8cyclictest16156-21df07:40:140
258899317,11cyclictest0-21swapper/010:56:500
258899313,26cyclictest0-21swapper/008:51:080
2588993126,4cyclictest0-21swapper/007:55:430
2591993028,1cyclictest0-21swapper/310:22:043
259199301,27cyclictest22370-21diskmemload09:17:193
259099301,5cyclictest3650-21diskmemload10:54:532
259099301,27cyclictest3561-21ssh11:18:192
258899307,9cyclictest11232-21unixbench_singl08:40:250
258899307,8cyclictest0-21swapper/011:14:140
259199290,27cyclictest32260-21taskset10:28:153
259099292,25cyclictest52850irq/140-enp3s0-11:14:432
259099291,26cyclictest300-21ssh10:28:452
2589992922,4cyclictest25166-21sh10:46:161
258999291,25cyclictest0-21swapper/109:37:251
258999290,26cyclictest31634-21ssh09:15:311
258899299,7cyclictest1900-21ntp_states08:20:200
258899295,9cyclictest0-21swapper/009:45:190
258899291,26cyclictest0-21swapper/008:47:230
258899291,26cyclictest0-21swapper/008:47:220
170482290,1sleep10-21swapper/112:15:361
2591992819,7cyclictest6166-21ssh10:32:453
259199280,5cyclictest13212-21ssh09:25:553
2590992821,5cyclictest20137-21ssh10:43:122
259099281,25cyclictest28537-21ssh10:49:352
2589992823,3cyclictest24625-21cpuspeed_turbos11:10:131
258999281,25cyclictest28285-21ssh10:00:551
258999281,25cyclictest0-21swapper/109:00:451
258999281,25cyclictest0-21swapper/108:49:551
258999281,25cyclictest0-21swapper/108:49:541
258999281,25cyclictest0-21swapper/107:54:331
258999280,25cyclictest22690-21ssh09:10:071
258899288,7cyclictest3193-21hddtemp_smartct07:10:170
2591992725,1cyclictest0-21swapper/311:18:363
2591992722,3cyclictest4041-21ntp_states08:25:203
259199272,18cyclictest21825-21runrttasks12:20:113
2591992719,6cyclictest22370-21diskmemload10:04:203
259199271,24cyclictest24058-21ssh11:33:053
259199271,24cyclictest0-21swapper/308:58:473
259099271,24cyclictest0-21swapper/208:45:352
259099271,24cyclictest0-21swapper/208:45:342
2589992724,2cyclictest0-21swapper/108:07:211
2589992720,5cyclictest22739-21expr07:55:131
258999271,24cyclictest0-21swapper/110:42:261
258999271,24cyclictest0-21swapper/109:08:341
258999271,24cyclictest0-21swapper/108:56:141
258999271,24cyclictest0-21swapper/108:25:171
258999271,1cyclictest0-21swapper/108:30:381
258899277,7cyclictest15160-31updatedb.mlocat07:38:040
258899276,8cyclictest0-21swapper/008:25:160
258899271,2cyclictest0-21swapper/008:19:040
258899271,24cyclictest0-21swapper/007:53:230
2591992622,3cyclictest34-21ksoftirqd/310:48:443
259199261,2cyclictest22370-21diskmemload09:35:203
259199261,23cyclictest0-21swapper/309:30:163
259199261,23cyclictest0-21swapper/309:30:163
259199261,23cyclictest0-21swapper/308:21:103
259199261,1cyclictest0-21swapper/310:15:183
259199261,1cyclictest0-21swapper/310:15:173
2590992623,2cyclictest0-21swapper/209:40:192
2590992615,7cyclictest52850irq/140-enp3s0-12:30:122
259099261,24cyclictest0-21swapper/210:04:262
259099261,23cyclictest0-21swapper/208:50:562
259099261,23cyclictest0-21swapper/208:19:552
259099261,23cyclictest0-21swapper/208:06:272
259099261,23cyclictest0-21swapper/207:54:172
2589992624,1cyclictest0-21swapper/108:19:051
2589992623,2cyclictest0-21swapper/110:22:031
2589992623,2cyclictest0-21swapper/110:15:011
2589992623,2cyclictest0-21swapper/108:50:511
258999261,24cyclictest0-21swapper/111:02:461
258999261,24cyclictest0-21swapper/110:57:011
258999261,24cyclictest0-21swapper/110:35:321
258999261,24cyclictest0-21swapper/109:33:371
258999261,24cyclictest0-21swapper/109:33:361
258999261,24cyclictest0-21swapper/109:21:131
2588992624,1cyclictest0-21swapper/008:30:510
2588992623,2cyclictest0-21swapper/008:02:410
2588992615,8cyclictest30967-21cron08:15:000
2588992610,14cyclictest21152-21sed09:05:180
104792260,1sleep10-21swapper/111:46:381
2591992523,1cyclictest0-21swapper/309:05:373
2591992522,2cyclictest0-21swapper/311:08:423
2591992522,2cyclictest0-21swapper/309:53:563
2591992522,2cyclictest0-21swapper/308:02:263
259199251,23cyclictest0-21swapper/311:11:083
259199251,23cyclictest0-21swapper/310:05:073
259199251,23cyclictest0-21swapper/308:20:003
259199251,23cyclictest0-21swapper/307:55:433
259199251,1cyclictest0-21swapper/308:50:013
259199251,1cyclictest0-21swapper/308:05:453
259199250,2cyclictest31705-21ssh10:50:483
259199250,23cyclictest0-21swapper/309:00:043
259199250,23cyclictest0-21swapper/308:30:513
2590992523,1cyclictest0-21swapper/209:00:022
2590992523,1cyclictest0-21swapper/208:57:422
259099251,23cyclictest0-21swapper/211:21:512
259099251,23cyclictest0-21swapper/209:37:112
259099251,23cyclictest0-21swapper/208:20:032
259099251,23cyclictest0-21swapper/207:57:252
259099250,23cyclictest6566-21ssh09:21:022
259099250,23cyclictest0-21swapper/208:44:582
2589992523,1cyclictest0-21swapper/111:20:211
2589992523,1cyclictest0-21swapper/110:31:451
2589992523,1cyclictest0-21swapper/110:15:011
2589992523,1cyclictest0-21swapper/110:15:001
2589992523,1cyclictest0-21swapper/108:03:071
258999251,1cyclictest0-21swapper/111:33:121
258999251,1cyclictest0-21swapper/109:52:321
258999250,1cyclictest0-21swapper/108:22:021
2588992516,6cyclictest18761-21missed_timers07:45:190
2588992514,8cyclictest12057-21tail07:30:180
2591992422,1cyclictest0-21swapper/309:40:163
2591992422,1cyclictest0-21swapper/309:20:453
2591992420,2cyclictest0-21swapper/308:40:193
2591992415,7cyclictest3644-21ntp_states12:30:193
259199241,1cyclictest0-21swapper/311:23:513
259199241,1cyclictest0-21swapper/309:58:193
259199241,1cyclictest0-21swapper/307:53:473
259199241,16cyclictest14309-21ssh11:50:033
259199241,16cyclictest14309-21ssh11:50:033
2590992422,1cyclictest0-21swapper/208:02:112
259099241,1cyclictest0-21swapper/209:33:372
259099241,1cyclictest0-21swapper/209:33:362
259099241,1cyclictest0-21swapper/209:05:412
259099241,1cyclictest0-21swapper/208:30:262
259099240,23cyclictest0-21swapper/211:26:442
259099240,23cyclictest0-21swapper/210:30:362
2589992417,5cyclictest7159-21apt-get08:35:021
2589992416,6cyclictest1062-21ssh12:28:531
2588992415,6cyclictest0-21swapper/007:15:110
2588992414,7cyclictest7487-21tail07:20:150
2591992321,1cyclictest0-21swapper/311:02:173
2591992321,1cyclictest0-21swapper/308:35:013
2590992321,1cyclictest23969-21ssh09:58:212
2590992321,1cyclictest0-21swapper/211:02:472
2590992318,3cyclictest13820-21ssh10:15:002
2590992318,3cyclictest13820-21ssh10:15:002
2589992320,2cyclictest0-21swapper/109:40:311
259099221,19cyclictest11189-21/usr/sbin/munin07:30:272
259199217,11cyclictest8237-21interrupts11:45:163
2591992111,7cyclictest560-21dbus-daemon12:20:013
259199211,17cyclictest278-21systemd-journal09:50:013
2590992119,1cyclictest0-21swapper/208:35:192
2590992110,9cyclictest22138-21cat12:20:142
2589992119,1cyclictest0-21swapper/109:58:141
2589992114,5cyclictest22-21ksoftirqd/107:39:591
258999210,2cyclictest141rcu_preempt11:55:251
11412210,3sleep228-21ksoftirqd/211:40:132
95002200,2chrt0-21swapper/107:25:151
2591992014,4cyclictest34-21ksoftirqd/308:15:003
2591992010,3cyclictest34-21ksoftirqd/307:15:153
2590992011,7cyclictest15819-21python311:50:242
2590992011,7cyclictest15819-21python311:50:232
258999203,4cyclictest141rcu_preempt11:44:321
258999202,12cyclictest8513-21apt-get12:10:111
2589992018,1cyclictest0-21swapper/108:40:151
2589992013,5cyclictest9603-21sh09:47:371
2589992012,6cyclictest5080-21ls07:15:131
2589992011,7cyclictest17816-21ls07:45:001
2589992011,6cyclictest12447-21date07:30:221
2589992010,8cyclictest20491-21ssh11:54:441
2589992010,8cyclictest20491-21ssh11:54:431
259199192,13cyclictest3591-21python312:05:233
2591991917,1cyclictest0-21swapper/310:37:263
259199191,5cyclictest27449-21ntp_states11:35:203
2591991913,4cyclictest34-21ksoftirqd/307:40:003
2591991912,5cyclictest7449-21expr10:10:133
2591991912,5cyclictest18774-21sed07:45:193
2591991911,6cyclictest7061-21expr07:20:123
2591991910,8cyclictest52950irq/141-enp3s0-12:03:253
259099191,7cyclictest4932-21chrt12:06:222
2590991917,1cyclictest0-21swapper/210:57:202
2590991913,4cyclictest32151-21sh12:27:322
2590991913,4cyclictest13068-21sh12:37:482
2590991913,4cyclictest13068-21sh12:37:472
2590991912,5cyclictest8508-21awk11:45:192
2590991911,6cyclictest12938-21seq12:13:152
2590991911,6cyclictest10229-21sensors_temp07:25:222
2590991910,6cyclictest278-21systemd-journal07:45:052
2589991914,4cyclictest22-21ksoftirqd/107:25:001
2589991914,3cyclictest4475-21sh12:30:341
2589991914,3cyclictest28900-21needreboot12:00:181
2589991910,6cyclictest26965-21gltestperf11:35:151
157862190,3chrt0-21swapper/307:40:013
106252190,1sleep20-21swapper/209:48:372
259199185,10cyclictest14870-21runrttasks12:39:423
259199185,10cyclictest14870-21runrttasks12:39:413
2591991813,3cyclictest2107-21sh11:40:223
2591991811,5cyclictest21588-21entropy11:55:153
2591991810,6cyclictest12551-21cat07:30:233
2590991813,3cyclictest24888-21sh11:57:232
2590991812,4cyclictest29901-21sh12:00:392
259099181,15cyclictest26908-21cut11:35:152
2590991810,6cyclictest6010-21ls07:15:222
2590991810,6cyclictest3459-21ls07:10:212
2589991813,3cyclictest2888-21hwlatdetect12:05:171
259199171,7cyclictest2609-21/usr/sbin/munin07:10:233
2591991712,3cyclictest8478-21/usr/sbin/munin12:10:123
259099171,11cyclictest15352-21apt-get07:40:042
2590991710,5cyclictest7863-21ntpq07:20:202
258999179,6cyclictest30581-21grep08:10:261
2589991711,4cyclictest18154-21expr07:45:121
2589991710,5cyclictest2737-21grep07:10:141
258999170,15cyclictest12692-21ssh12:37:251
258999170,15cyclictest12692-21ssh12:37:241
259199161,11cyclictest5723-21timerandwakeup10:55:273
259099169,5cyclictest29765-21sed08:10:182
259099169,5cyclictest14522-21awk07:35:202
259199158,5cyclictest10320-21timerandwakeup07:25:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional