You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 19:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Dec 06, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
197372121110,4sleep3411rcuc/309:25:133
1623991090,4rtkit-daemon0-21swapper/010:58:150
1623991090,4rtkit-daemon0-21swapper/010:58:150
20010242930,6sleep02197014-21PK-Backend10:40:170
162399900,69rtkit-daemon0-21swapper/307:07:463
162399860,6rtkit-daemon2197014-21PK-Backend11:07:492
20379852780,5sleep10-21swapper/112:25:021
20303892730,4chrt2030390-21apt-config12:05:021
192462299732,12cyclictest1753-21gnome-shell09:45:232
192446227248,16sleep10-21swapper/107:09:531
192415127152,11sleep00-21swapper/007:05:500
192415326839,23sleep20-21swapper/207:05:522
162399680,5rtkit-daemon2197014-21PK-Backend08:41:350
162399680,50rtkit-daemon921-21thermald11:59:091
162399680,10rtkit-daemon2284-21update-notifier11:33:130
1924622996521,19cyclictest34-21ksoftirqd/210:45:012
20140562640,7sleep02197014-21PK-Backend11:15:310
20140562640,7sleep02197014-21PK-Backend11:15:300
1924622995713,9cyclictest34-21ksoftirqd/207:40:142
1924615995719,11cyclictest1988954-21grep10:05:310
1924615995620,10cyclictest1983636-21cat09:50:300
1924615995618,12cyclictest2023455-21date11:45:020
1924615995617,11cyclictest1986376-21cat10:00:150
19949102550,6chrt2197014-21PK-Backend10:21:272
192462299556,10cyclictest34-21ksoftirqd/211:38:522
162399550,41rtkit-daemon42-21ksoftirqd/308:41:243
19571582540,8chrt0-21swapper/008:35:330
1924615995420,10cyclictest13-21ksoftirqd/008:00:480
162399540,8rtkit-daemon1622-21rtkit-daemon11:54:571
1924622995317,8cyclictest34-21ksoftirqd/210:34:462
1924615995321,13cyclictest2034050-21/usr/sbin/munin12:15:010
1924615995319,11cyclictest2020973-21sensors11:35:300
192462799521,17cyclictest2026837-21munin-run11:55:013
19697572510,6chrt0-21swapper/209:15:032
19263202510,15sleep31908729-21kworker/3:107:15:013
1924622995119,10cyclictest34-21ksoftirqd/210:00:012
1924615995119,10cyclictest2040391-21sed12:30:240
1924615995116,13cyclictest2002819-21df10:45:150
162399510,8rtkit-daemon1622-21rtkit-daemon12:07:531
1924627995027,10cyclictest42-21ksoftirqd/310:35:013
1924622995018,9cyclictest34-21ksoftirqd/207:55:002
1924622995011,21cyclictest34-21ksoftirqd/208:15:002
1924615995019,11cyclictest13-21ksoftirqd/009:21:110
162399500,37rtkit-daemon2000474-21idleruntime-cro10:40:012
1924627994927,9cyclictest42-21ksoftirqd/311:38:523
192462299498,5cyclictest141rcu_preempt08:30:072
192462299498,5cyclictest141rcu_preempt08:30:072
192462299497,26cyclictest141rcu_preempt10:15:282
1924622994938,8cyclictest34-21ksoftirqd/211:05:012
192462799481,13cyclictest2197014-21PK-Backend10:30:013
192462299482,5cyclictest141rcu_preempt09:40:012
1924622994819,10cyclictest34-21ksoftirqd/209:20:012
1924622994815,10cyclictest34-21ksoftirqd/209:45:012
1924615994822,21cyclictest1938215-40lsb_release07:45:200
1924627994714,24cyclictest42-21ksoftirqd/307:45:103
1924627994712,5cyclictest141rcu_preempt10:40:023
1924622994736,8cyclictest34-21ksoftirqd/207:20:002
1924622994735,9cyclictest34-21ksoftirqd/207:40:022
1924622994720,10cyclictest34-21ksoftirqd/210:50:042
1924622994719,9cyclictest34-21ksoftirqd/209:25:022
1924622994710,19cyclictest34-21ksoftirqd/210:45:132
1924618994726,16cyclictest3242253-21nautilus10:30:011
162399470,25rtkit-daemon1622-21rtkit-daemon08:41:561
192462799466,21cyclictest141rcu_preempt09:40:293
1924627994614,24cyclictest42-21ksoftirqd/310:24:003
192462299468,20cyclictest34-21ksoftirqd/211:45:022
1924622994625,8cyclictest34-21ksoftirqd/210:03:242
1924622994612,10cyclictest34-21ksoftirqd/209:25:072
192461899463,39cyclictest1622-21rtkit-daemon09:12:511
1924618994626,15cyclictest3242253-21nautilus10:20:021
1924618994625,18cyclictest855-21dbus-daemon12:35:011
1924615994620,8cyclictest1753-21gnome-shell08:59:080
162399460,5rtkit-daemon0-21swapper/310:15:093
192462799456,26cyclictest1908729-21kworker/3:112:30:193
1924627994521,21cyclictest2007726-21unin-run11:00:013
1924627994521,21cyclictest2007726-21unin-run11:00:013
192462299452,22cyclictest1962380-21chrt08:50:392
1924622994517,10cyclictest34-21ksoftirqd/207:10:212
192462299451,40cyclictest1753-21gnome-shell08:16:152
1924622994513,15cyclictest34-21ksoftirqd/210:12:572
1924622994510,11cyclictest34-21ksoftirqd/211:45:292
1924627994422,9cyclictest42-21ksoftirqd/307:20:003
1924627994420,10cyclictest42-21ksoftirqd/309:40:013
192462299449,19cyclictest34-21ksoftirqd/209:30:242
192462299449,11cyclictest34-21ksoftirqd/207:45:252
192462299447,20cyclictest34-21ksoftirqd/211:35:012
1924622994418,7cyclictest34-21ksoftirqd/208:38:022
1924622994415,10cyclictest34-21ksoftirqd/211:20:232
1924622994414,10cyclictest34-21ksoftirqd/209:50:162
1924622994411,10cyclictest34-21ksoftirqd/211:10:152
1924618994421,18cyclictest1999544-21grep10:35:231
192461899440,41cyclictest1622-21rtkit-daemon08:45:521
162399440,37rtkit-daemon1389-21kerneloops11:36:501
192462799439,15cyclictest42-21ksoftirqd/311:24:533
192462799438,16cyclictest42-21ksoftirqd/308:29:093
1924627994323,13cyclictest42-21ksoftirqd/308:50:303
1924627994322,14cyclictest42-21ksoftirqd/308:03:513
1924627994321,14cyclictest42-21ksoftirqd/309:00:313
1924627994319,20cyclictest1753-21gnome-shell12:06:263
1924627994311,20cyclictest141rcu_preempt09:05:133
192462799431,10cyclictest1955825-21awk08:35:013
192462799431,10cyclictest1955825-21awk08:35:013
1924627994310,16cyclictest42-21ksoftirqd/311:05:313
192462299435,20cyclictest34-21ksoftirqd/212:25:262
192462299434,19cyclictest34-21ksoftirqd/208:55:142
1924622994314,10cyclictest34-21ksoftirqd/212:25:012
1924622994312,9cyclictest34-21ksoftirqd/208:25:312
1924622994312,10cyclictest34-21ksoftirqd/208:21:292
1924622994312,10cyclictest34-21ksoftirqd/208:00:182
1924622994311,9cyclictest34-21ksoftirqd/207:55:232
192462299431,17cyclictest20142072/usr/sbin/munin11:15:312
192462299431,17cyclictest20142072/usr/sbin/munin11:15:312
1924622994311,10cyclictest34-21ksoftirqd/210:05:302
1924622994311,10cyclictest34-21ksoftirqd/208:07:442
1924622994311,10cyclictest34-21ksoftirqd/208:07:432
162399430,36rtkit-daemon881143-21gmain11:28:171
1924627994225,7cyclictest1753-21gnome-shell12:02:573
1924627994225,12cyclictest1991141-21apt-get10:15:013
192462799421,23cyclictest1753-21gnome-shell12:14:533
1924622994216,8cyclictest34-21ksoftirqd/212:05:012
162399420,35rtkit-daemon856-21gmain09:57:071
162399420,31rtkit-daemon2197014-21PK-Backend10:57:152
162399420,31rtkit-daemon2197014-21PK-Backend10:57:152
162399420,30rtkit-daemon0-21swapper/211:54:332
20142082410,4sleep10-21swapper/111:15:311
20142082410,4sleep10-21swapper/111:15:311
19824812410,6sleep0161rcuc/009:50:020
192462799417,9cyclictest42-21ksoftirqd/311:16:003
192462799417,9cyclictest42-21ksoftirqd/311:16:003
1924627994124,13cyclictest42-21ksoftirqd/312:36:153
1924627994123,6cyclictest42-21ksoftirqd/311:30:123
1924627994122,6cyclictest42-21ksoftirqd/311:45:343
1924627994113,11cyclictest141rcu_preempt08:15:173
1924627994111,17cyclictest42-21ksoftirqd/308:20:203
192462299418,18cyclictest34-21ksoftirqd/212:35:432
1924622994129,8cyclictest331rcuc/208:45:012
1924622994125,12cyclictest2028546-21/usr/sbin/munin12:00:012
1924622994116,7cyclictest34-21ksoftirqd/208:47:162
1924622994115,8cyclictest34-21ksoftirqd/210:29:302
1924618994126,11cyclictest2003883-21sshd10:49:431
1924618994124,12cyclictest2043010-21cron12:40:001
1924618994123,14cyclictest3242253-21nautilus08:15:011
1924618994122,14cyclictest1941753-21interrupts07:55:221
1924615994123,13cyclictest1-21systemd08:20:010
192462799407,15cyclictest42-21ksoftirqd/311:11:363
1924627994027,8cyclictest1753-21gnome-shell09:18:573
1924627994027,10cyclictest42-21ksoftirqd/309:25:013
1924627994023,13cyclictest42-21ksoftirqd/311:55:243
1924627994022,6cyclictest42-21ksoftirqd/310:05:153
1924627994018,9cyclictest42-21ksoftirqd/311:04:243
1924627994014,6cyclictest141rcu_preempt12:15:253
1924627994011,16cyclictest42-21ksoftirqd/308:09:523
1924627994011,16cyclictest42-21ksoftirqd/308:09:523
1924627994011,11cyclictest141rcu_preempt07:35:133
192462299408,18cyclictest34-21ksoftirqd/212:34:402
1924622994022,13cyclictest2031671-21sensors_temp12:05:292
192462299402,18cyclictest1753-21gnome-shell12:10:132
1924622994021,14cyclictest2017405-21ntpq11:25:412
1924618994021,15cyclictest2018776-21hddtemp_smartct11:30:211
1924618994021,14cyclictest1980919-40apt-check09:45:181
1924618994020,16cyclictest2010852-21sshd11:06:161
1924627993925,10cyclictest42-21ksoftirqd/307:30:133
192462799392,33cyclictest2022461-21memory11:40:233
192462799392,33cyclictest1753-21gnome-shell08:57:413
1924627993921,13cyclictest42-21ksoftirqd/307:20:283
1924627993917,9cyclictest42-21ksoftirqd/308:48:453
1924627993916,10cyclictest42-21ksoftirqd/310:53:343
192462799391,24cyclictest42-21ksoftirqd/307:50:303
1924627993910,16cyclictest42-21ksoftirqd/309:11:483
1924627993910,12cyclictest141rcu_preempt12:20:133
1924622993928,8cyclictest34-21ksoftirqd/207:26:432
1924622993920,7cyclictest34-21ksoftirqd/212:18:122
192461899392,9cyclictest859-21NetworkManager10:34:421
1924618993928,8cyclictest3242254-21gmain10:21:131
1924618993922,12cyclictest438-20systemd-journal09:16:041
1924618993922,12cyclictest438-20systemd-journal08:54:551
1924618993921,13cyclictest1945786-21ntpq08:05:251
1924618993921,13cyclictest1945786-21ntpq08:05:241
1924618993919,15cyclictest3242253-21nautilus10:45:011
1924615993928,7cyclictest1753-21gnome-shell12:28:590
1924615993922,13cyclictest1951268-21ntp_states08:20:270
1924615993922,12cyclictest1954271-40apt-check08:30:170
1924615993922,12cyclictest1954271-40apt-check08:30:170
1924627993822,13cyclictest42-21ksoftirqd/307:27:213
1924627993821,6cyclictest42-21ksoftirqd/312:25:423
1924627993821,5cyclictest42-21ksoftirqd/309:50:223
1924627993816,9cyclictest42-21ksoftirqd/308:35:213
1924627993816,9cyclictest42-21ksoftirqd/308:10:253
1924627993814,6cyclictest141rcu_preempt10:02:343
1924627993813,9cyclictest42-21ksoftirqd/311:25:163
192462299381,18cyclictest34-21ksoftirqd/207:30:212
1924622993811,10cyclictest34-21ksoftirqd/209:01:062
1924618993822,12cyclictest1966870-21fw_packets09:05:201
1924618993821,12cyclictest438-20systemd-journal08:20:001
1924618993820,13cyclictest1954468-21cut08:30:171
1924618993820,13cyclictest1954468-21cut08:30:161
1924618993819,14cyclictest3242253-21nautilus12:20:031
1924615993825,9cyclictest1753-21gnome-shell08:13:580
192461599382,32cyclictest1753-21gnome-shell10:54:420
1924615993822,11cyclictest438-20systemd-journal09:16:430
1924615993821,12cyclictest2030277-21/usr/sbin/munin12:05:020
1924615993819,15cyclictest2034305-21cat12:15:110
1924615993818,15cyclictest3242253-21nautilus10:25:020
192461599381,32cyclictest2011296-21cat11:10:020
19754142372,9sleep31753-21gnome-shell09:30:063
1924627993726,7cyclictest1753-21gnome-shell10:49:113
1924627993721,13cyclictest42-21ksoftirqd/310:40:043
1924627993721,12cyclictest42-21ksoftirqd/307:40:343
192462799371,6cyclictest2197014-21PK-Backend09:55:213
1924627993714,10cyclictest42-21ksoftirqd/307:56:443
1924627993710,16cyclictest42-21ksoftirqd/309:47:363
192462299372,7cyclictest1753-21gnome-shell09:10:022
1924622993710,10cyclictest34-21ksoftirqd/207:20:302
1924618993726,8cyclictest26-21ksoftirqd/110:03:261
1924618993723,9cyclictest26-21ksoftirqd/107:45:021
1924618993720,13cyclictest1951830-21sshd08:22:431
1924618993720,13cyclictest1940974-21sshd07:54:431
192461899371,27cyclictest0-21swapper/107:25:351
1924618993712,19cyclictest1753-21gnome-shell08:00:161
1924618993711,22cyclictest2181-21gdbus10:50:031
1924615993727,6cyclictest2015786-21switchtime11:20:300
1924615993725,7cyclictest1937250-21turbostat07:45:000
1924615993723,10cyclictest1996750-21sshd10:27:430
192461599371,32cyclictest1753-21gnome-shell12:36:390
1924615993713,20cyclictest1933598-21ade-motd07:33:430
1924618993620,12cyclictest1965857-21seq09:00:471
1924618993620,11cyclictest438-20systemd-journal12:25:421
1924618993617,15cyclictest1978652-21sh09:40:001
192461599362,8cyclictest1753-21gnome-shell07:37:440
1924615993627,6cyclictest0-21swapper/007:56:470
1924615993626,6cyclictest1969681-21turbostat09:15:000
1924615993625,7cyclictest1753-21gnome-shell09:31:310
1924615993620,12cyclictest1977315-21expr09:35:160
1924615993620,12cyclictest1961680-21hddtemp_smartct08:50:240
1924615993620,11cyclictest1954059-21dpkg08:30:020
1924615993619,12cyclictest1946404-21seq08:08:430
1924615993619,12cyclictest1946404-21seq08:08:430
1924615993619,12cyclictest1940043-21kernelversion07:50:220
1924615993618,13cyclictest2011852-21cat11:10:200
1924618993524,7cyclictest2011559-21cpuspeed_turbos11:10:141
1924618993524,7cyclictest1963672-21needreboot08:55:251
1924618993523,8cyclictest1937777-40apt-check07:45:231
1924618993521,10cyclictest1753-21gnome-shell07:35:281
1924618993520,11cyclictest1928100-21cat07:15:311
1924618993519,12cyclictest2021873-21awk11:40:121
1924618993519,12cyclictest1952293-21apt08:25:121
1924618993518,13cyclictest1-21systemd11:00:011
1924618993518,13cyclictest1-21systemd11:00:011
1924618993517,14cyclictest1957542-21latency_hist08:40:001
1924615993524,7cyclictest1984525-21cpuspeed_turbos09:55:140
1924615993524,7cyclictest1753-21gnome-shell09:43:240
1924615993524,7cyclictest1753-21gnome-shell07:20:190
1924615993522,9cyclictest1753-21gnome-shell09:06:140
1924615993520,11cyclictest2024231-21wc11:45:260
1924615993520,11cyclictest1997506-21cat10:30:160
1924615993519,12cyclictest2008820-21rt-features11:00:300
1924615993518,13cyclictest1999712-21ntpq10:35:250
1924615993517,13cyclictest1931593-21users07:25:320
192461599351,10cyclictest2032318-21turbostat12:09:590
162399350,29rtkit-daemon1924610-21cyclictest10:10:011
162399350,29rtkit-daemon0-21swapper/111:01:151
1924618993419,11cyclictest1990444-21awk10:10:271
1924618993418,12cyclictest2015077-21grep11:20:141
1924618993418,12cyclictest1983213-21expr09:50:221
1924618993417,12cyclictest2033143-21cat12:10:211
1924618993417,12cyclictest1980417-21pluginstate09:45:011
192461899341,29cyclictest1973548-21apt-config09:25:031
1924615993424,7cyclictest0-21swapper/007:10:290
1924615993421,8cyclictest1753-21gnome-shell08:46:090
192461599342,14cyclictest1992422-40lsb_release10:15:260
1924615993419,11cyclictest2038000-21cat12:25:020
1924615993419,11cyclictest2028110-21cat11:55:310
1924615993418,12cyclictest2018128-21pool-org.gnome.11:30:010
1924615993418,12cyclictest1927821-21awk07:15:270
1924615993417,13cyclictest1965348-21cat09:00:250
20257962330,5chrt2197014-21PK-Backend11:50:200
1924618993322,7cyclictest1069-21runrttasks07:30:231
1924618993321,8cyclictest2023824-21cstates11:45:171
1924618993318,11cyclictest1971893-21expr09:20:141
1924618993316,12cyclictest1976203-21ntpq09:30:311
1924615993318,11cyclictest3242253-21nautilus09:25:040
1924618993217,11cyclictest1925866-21grep07:10:311
1924615993217,11cyclictest1991169-21gpgv10:15:010
1924618993117,10cyclictest1929240-21cat07:20:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional