You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 03:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Wed Jul 29, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1169521690,5sleep02177799cyclictest22:43:010
2731721670,5sleep32178099cyclictest21:42:233
21777991314,37cyclictest21849-21ssh22:15:470
217779912433,12cyclictest13480-21/usr/sbin/munin00:05:550
217779912433,12cyclictest13480-21/usr/sbin/munin00:05:550
21777991093,71cyclictest0-21swapper/022:00:540
2233121020,4sleep031446-21kworker/0:200:15:490
2177799944,56cyclictest0-21swapper/021:35:480
2177799944,56cyclictest0-21swapper/021:35:480
2177799933,56cyclictest0-21swapper/021:25:360
21777999038,3cyclictest0-21swapper/022:05:360
21777999038,3cyclictest0-21swapper/022:05:360
309712890,6sleep32178099cyclictest19:35:263
2177799882,50cyclictest0-21swapper/023:40:370
21777998749,3cyclictest7950irq/56-dwc_otg_00:10:380
21777998643,20cyclictest750-21snmpd00:35:430
21778998369,9cyclictest10296-21ls22:00:551
58452790,12sleep12177899cyclictest00:35:441
21780997928,22cyclictest8050irq/56-dwc_otg_22:15:473
21779997910,27cyclictest7950irq/56-dwc_otg_00:35:442
21780997749,22cyclictest14336-21/usr/sbin/munin00:05:553
21780997749,22cyclictest14336-21/usr/sbin/munin00:05:553
2177899775,38cyclictest0-21swapper/122:55:381
111122750,2sleep00-21swapper/020:10:250
21780997446,23cyclictest7084-21ssh23:17:013
21780997416,52cyclictest0-21swapper/323:50:353
21780997326,19cyclictest8050irq/56-dwc_otg_00:35:413
2177999736,19cyclictest8050irq/56-dwc_otg_19:55:332
2177799736,25cyclictest0-21swapper/019:40:380
2177799733,49cyclictest0-21swapper/023:50:360
21779997115,25cyclictest7850irq/56-dwc_otg22:15:462
54062700,2sleep30-21swapper/321:15:493
2177799705,20cyclictest0-21swapper/023:17:000
26712690,2sleep2301ktimersoftd/221:12:102
19892690,2sleep10-21swapper/121:10:561
21780996837,25cyclictest21931-21ssh21:35:503
21780996837,25cyclictest21931-21ssh21:35:503
2177899683,44cyclictest0-21swapper/122:15:451
21780996729,21cyclictest30381-21/usr/sbin/munin23:45:523
2177899678,51cyclictest0-21swapper/100:10:381
2177799671,37cyclictest0-21swapper/019:25:360
21780996637,23cyclictest10223-21sort22:00:543
21780996625,17cyclictest31669-21ls19:35:473
2177799662,35cyclictest30196-21fschecks_time00:25:440
21780996517,29cyclictest27549-21ssh23:02:253
2177799644,33cyclictest0-21swapper/019:20:560
21777996411,30cyclictest9-21ksoftirqd/023:15:260
2177799632,30cyclictest0-21swapper/000:35:260
21777996235,14cyclictest8050irq/56-dwc_otg_20:25:250
2177899617,30cyclictest14309-21ntpq00:05:541
2177899617,30cyclictest14309-21ntpq00:05:541
21778996111,27cyclictest7950irq/56-dwc_otg_21:35:481
21778996111,27cyclictest7950irq/56-dwc_otg_21:35:481
21777996132,5cyclictest7950irq/56-dwc_otg_19:20:260
21780996015,28cyclictest8050irq/56-dwc_otg_21:20:523
2177799604,31cyclictest0-21swapper/022:25:530
21777996040,4cyclictest17281-21/usr/sbin/munin22:50:440
21777996031,6cyclictest21527-21wget22:55:410
21780995920,26cyclictest25614-21apt-get19:20:323
21780995920,24cyclictest20875-21chrt23:34:563
21780995920,19cyclictest7850irq/56-dwc_otg19:15:263
21780995918,23cyclictest25349-21latency_hist23:40:263
21779995912,31cyclictest0-21swapper/222:00:552
2177799596,23cyclictest29354-21ssh21:45:330
2177799592,35cyclictest1327-21ssh21:50:450
2177799592,30cyclictest5106-21sort21:15:460
2177799591,3cyclictest7850irq/56-dwc_otg23:25:360
2177799582,29cyclictest26485-21grep21:40:560
21777995821,19cyclictest21755-21fschecks_count23:35:440
21780995723,17cyclictest7359-21ssh22:37:323
21780995721,23cyclictest28498-21timerandwakeup20:55:543
21780995720,17cyclictest22416-21ssh00:15:503
21780995716,13cyclictest28688-21ssh00:23:553
21780995716,13cyclictest28688-21ssh00:23:553
21779995711,32cyclictest0-21swapper/200:05:522
21779995711,32cyclictest0-21swapper/200:05:522
21777995736,6cyclictest7950irq/56-dwc_otg_21:05:250
2177799572,6cyclictest1133-21gpg-connect-age23:50:270
2177799572,31cyclictest10338-21ssh23:20:560
21780995621,8cyclictest2083-21ls21:50:543
21780995620,21cyclictest31676-21ssh21:47:453
21780995617,23cyclictest29223-21ssh23:45:213
21777995638,4cyclictest25614-21ls22:20:400
2177799563,29cyclictest0-21swapper/019:50:450
2177799562,36cyclictest0-21swapper/021:56:440
21777995615,27cyclictest9650irq/80-mmc022:15:250
2177799561,34cyclictest6623-21grep23:55:560
21780995526,12cyclictest16001-21ls20:20:513
21779995510,10cyclictest31-21ksoftirqd/221:35:492
21779995510,10cyclictest31-21ksoftirqd/221:35:492
2177899558,29cyclictest0-21swapper/119:20:311
2177899553,31cyclictest0-21swapper/123:17:021
21777995530,5cyclictest13591-21fschecks_count22:45:450
2177799552,38cyclictest0-21swapper/022:30:510
2177799552,32cyclictest0-21swapper/020:15:470
21777995510,9cyclictest0-21swapper/019:35:370
21780995422,14cyclictest31455-21ssh00:26:303
21780995411,28cyclictest1291-21ssh23:10:403
2177799548,21cyclictest0-21swapper/019:55:330
2177799542,31cyclictest0-21swapper/022:35:470
2177799542,26cyclictest25042-21unixbench-2d20:45:540
21777995411,22cyclictest19076-21latency20:30:450
21780995322,11cyclictest4168-21ssh21:14:543
21780995321,14cyclictest27658-21diskstats19:25:423
21780995317,20cyclictest25937-21uname22:20:453
21780995317,10cyclictest7950irq/56-dwc_otg_21:31:283
2177999539,28cyclictest0-21swapper/219:20:312
21780995217,21cyclictest17851-21df00:10:403
21780995217,12cyclictest11454-21taskset00:02:143
21780995214,23cyclictest17283-21ssh22:50:353
2177999523,26cyclictest0-21swapper/223:50:352
21777995228,7cyclictest7950irq/56-dwc_otg_20:00:490
21777995222,17cyclictest18144-21ssh23:30:480
2177799521,30cyclictest1432-21ntpq21:10:490
2177799521,24cyclictest913250ssh21:20:420
21780995124,12cyclictest10903-21ssh23:21:313
21780995115,26cyclictest15870-21sh22:48:173
21780995114,12cyclictest0-21swapper/323:29:293
21779995128,12cyclictest7850irq/56-dwc_otg23:15:402
2177799513,27cyclictest0-21swapper/000:20:530
2177799513,27cyclictest0-21swapper/000:20:530
21777995121,21cyclictest0-21swapper/023:02:500
21777995118,16cyclictest2328-21wget19:45:420
21780995019,20cyclictest7289-21ssh23:57:053
21780995019,11cyclictest7850irq/56-dwc_otg22:09:453
21780995019,11cyclictest7850irq/56-dwc_otg22:09:453
21780995019,11cyclictest19604-21ssh22:12:513
21780995018,16cyclictest7950irq/56-dwc_otg_23:07:343
21780995018,16cyclictest7558-21ssh21:58:053
21780995017,11cyclictest4753-21ssh00:34:423
2177899504,19cyclictest0-21swapper/123:50:341
2177799507,28cyclictest0-21swapper/021:35:260
2177799506,8cyclictest0-21swapper/023:05:400
2177799506,25cyclictest0-21swapper/021:05:380
2178099498,27cyclictest0-21swapper/319:40:383
21780994919,10cyclictest26673-21cat20:50:543
21780994918,17cyclictest27264-1kworker/u9:122:30:403
2178099491,6cyclictest0-21swapper/319:55:383
21780994916,18cyclictest8050irq/56-dwc_otg_21:25:593
21780994912,12cyclictest14053-21/usr/sbin/munin20:15:543
2177799493,25cyclictest0-21swapper/000:03:010
214752493,12sleep00-21swapper/019:10:000
2178099484,26cyclictest8050irq/56-dwc_otg_20:35:433
21780994811,11cyclictest8050irq/56-dwc_otg_22:43:103
2177799481,26cyclictest0-21swapper/020:50:460
2177799481,24cyclictest0-21swapper/020:55:380
21780994713,13cyclictest23056-21ssh22:56:343
21780994710,21cyclictest8050irq/56-dwc_otg_20:31:383
2177999477,21cyclictest0-21swapper/219:10:422
2177799477,25cyclictest0-21swapper/020:40:390
2178099468,23cyclictest0-21swapper/321:00:383
2177999469,27cyclictest21776-21cyclictest21:45:312
21778994613,19cyclictest0-21swapper/122:20:371
2178099457,23cyclictest0-21swapper/320:00:373
21779994520,5cyclictest727-21runrttasks22:45:552
2177899457,23cyclictest0-21swapper/119:35:381
2177899454,22cyclictest0-21swapper/122:30:261
2177799452,21cyclictest0-21swapper/019:30:500
2178099448,9cyclictest0-21swapper/320:42:473
2178099447,24cyclictest0-21swapper/319:50:393
21780994415,18cyclictest8050irq/56-dwc_otg_22:30:113
21779994436,5cyclictest21430-21sh22:55:392
2177999442,24cyclictest29643-21ssh23:45:412
2177899447,17cyclictest13226-21ssh21:25:431
2177799446,25cyclictest0-21swapper/020:25:390
21777994417,5cyclictest22130-21fschecks_time19:10:460
21780994316,12cyclictest0-21swapper/319:15:263
2177999438,24cyclictest0-21swapper/200:10:382
2177999438,19cyclictest26255-21/usr/sbin/munin23:40:482
2177899437,24cyclictest0-21swapper/120:00:351
2177899437,19cyclictest0-21swapper/123:45:421
21778994310,5cyclictest0-21swapper/121:50:541
21777994324,5cyclictest7850irq/56-dwc_otg20:35:380
21780994225,13cyclictest3183-21sed19:45:553
21780994211,18cyclictest0-21swapper/320:28:573
2177999426,13cyclictest8050irq/56-dwc_otg_21:50:572
21779994225,9cyclictest7950irq/56-dwc_otg_21:20:502
21779994224,9cyclictest8050irq/56-dwc_otg_22:37:092
21779994218,10cyclictest8050irq/56-dwc_otg_22:51:242
21779994210,13cyclictest7850irq/56-dwc_otg23:56:482
2177899425,9cyclictest7950irq/56-dwc_otg_00:15:381
2177899423,25cyclictest0-21swapper/122:10:391
21780994112,11cyclictest32477-21chrt21:07:083
2177999417,24cyclictest0-21swapper/220:50:412
2177999414,19cyclictest8050irq/56-dwc_otg_00:15:502
21779994124,9cyclictest7950irq/56-dwc_otg_19:25:382
21779994118,5cyclictest7356-21sh21:57:412
21779994117,8cyclictest7850irq/56-dwc_otg23:07:292
21778994131,7cyclictest29009-21ls21:45:251
2177799413,35cyclictest0-21swapper/020:10:470
212552410,2sleep10-21swapper/123:35:261
2178099407,21cyclictest0-21swapper/320:45:403
2177999402,23cyclictest0-21swapper/220:30:392
21779994021,3cyclictest8050irq/56-dwc_otg_22:40:382
21779994016,9cyclictest8050irq/56-dwc_otg_00:32:212
2177899407,8cyclictest9108-21sh00:00:251
2177899406,23cyclictest25537-21ssh23:40:381
2177899405,23cyclictest9943-21ssh23:20:491
2177899402,8cyclictest25724-21diskstats23:00:441
21778994019,5cyclictest13807-21kernelversion22:45:481
225142390,2sleep10-21swapper/119:10:521
21780993918,10cyclictest11312-21sshd20:10:293
2177999394,4cyclictest7950irq/56-dwc_otg_23:40:252
2177999392,10cyclictest7850irq/56-dwc_otg21:35:252
21778993919,5cyclictest29537-21sh23:05:411
2177899391,27cyclictest26314-21latency00:20:471
2177899391,27cyclictest26314-21latency00:20:471
21779993821,12cyclictest8050irq/56-dwc_otg_23:00:392
21779993814,16cyclictest8050irq/56-dwc_otg_23:34:522
2177899386,17cyclictest7950irq/56-dwc_otg_22:50:521
2177899382,17cyclictest7950irq/56-dwc_otg_23:15:051
2177899382,16cyclictest7850irq/56-dwc_otg21:35:261
21778993819,5cyclictest29825-21/usr/sbin/munin00:25:501
21778993816,5cyclictest6289-21timerwakeupswit22:35:521
21778993815,5cyclictest29022-21idleruntime-cro19:30:251
2177899381,24cyclictest29689-21ssh21:45:441
21780993723,11cyclictest9786-21/usr/sbin/munin20:05:423
2177999379,17cyclictest301ktimersoftd/221:25:532
2177999379,13cyclictest8050irq/56-dwc_otg_21:16:402
2177999374,22cyclictest31660-21sh00:26:552
21779993713,16cyclictest31-21ksoftirqd/223:21:302
2177999371,25cyclictest12326-21ls20:10:532
2177999371,12cyclictest8050irq/56-dwc_otg_00:02:382
21778993721,7cyclictest726450ssh21:18:071
2177899371,11cyclictest9790-21sort20:05:421
226362363,4sleep10-21swapper/120:40:461
21779993617,9cyclictest8050irq/56-dwc_otg_22:20:422
21779993613,14cyclictest8050irq/56-dwc_otg_00:21:152
21779993613,14cyclictest8050irq/56-dwc_otg_00:21:152
2177999361,26cyclictest13640-21ls20:15:472
2177999361,24cyclictest4891-21ssh23:15:262
2177899368,5cyclictest32764-21idleruntime-cro21:10:251
21778993616,5cyclictest23-21ksoftirqd/120:50:391
21778993615,5cyclictest2358-21/usr/sbin/munin22:30:561
21778993610,4cyclictest7850irq/56-dwc_otg00:30:361
2177999358,5cyclictest31734-21ntp_states19:35:482
2177999356,6cyclictest29155-21apt-key21:45:262
21779993516,7cyclictest7950irq/56-dwc_otg_22:31:432
21779993512,11cyclictest8050irq/56-dwc_otg_23:29:252
21779993511,8cyclictest8050irq/56-dwc_otg_19:30:392
21779993511,5cyclictest29511-21cut22:25:382
2177999350,12cyclictest14946-21/usr/sbin/munin20:20:542
2177899353,8cyclictest5957-21wget19:55:381
2177899352,22cyclictest13499-21grep22:05:431
2177899352,22cyclictest13499-21grep22:05:431
2177999347,5cyclictest8050irq/56-dwc_otg_21:05:392
2177999347,24cyclictest0-21swapper/220:55:412
2177999347,22cyclictest0-21swapper/220:00:372
21779993415,9cyclictest8050irq/56-dwc_otg_22:10:322
2177899348,5cyclictest7850irq/56-dwc_otg21:20:411
2177899345,14cyclictest8050irq/56-dwc_otg_19:15:451
2177899343,4cyclictest7850irq/56-dwc_otg22:40:431
21778993411,6cyclictest0-21swapper/121:55:471
21778993410,7cyclictest22005-21ssh23:35:471
2177999337,23cyclictest0-21swapper/220:45:402
2177999333,5cyclictest7950irq/56-dwc_otg_19:15:262
2177899333,21cyclictest4969-21cat19:50:531
2177899331,18cyclictest0-21swapper/123:27:011
2177999328,12cyclictest8050irq/56-dwc_otg_22:05:492
2177999328,12cyclictest8050irq/56-dwc_otg_22:05:492
2177999327,22cyclictest0-21swapper/219:50:392
2177999327,14cyclictest0-21swapper/220:35:392
2177999322,6cyclictest10208-21tr20:05:472
2177899328,5cyclictest7850irq/56-dwc_otg20:15:541
2177899326,15cyclictest0-21swapper/120:20:381
21778993218,5cyclictest366-21/usr/sbin/munin19:40:511
21778993213,5cyclictest20562-21awk20:35:401
21778993212,6cyclictest0-21swapper/120:55:441
21778993211,5cyclictest9793-21tune2fs00:00:431
2177999311,7cyclictest2553-21fschecks_time19:45:452
2177899312,6cyclictest3372-21grep19:45:571
2177899312,6cyclictest29443-21wget21:00:381
2177999302,16cyclictest0-21swapper/220:45:252
2177899304,13cyclictest8050irq/56-dwc_otg_20:30:541
2177999291,24cyclictest0-21swapper/219:40:382
2177899293,6cyclictest17746-21ls20:25:531
2177899291,22cyclictest0-21swapper/120:11:261
205692290,2sleep30-21swapper/319:05:453
2177999287,19cyclictest0-21swapper/221:00:382
2177999285,11cyclictest8050irq/56-dwc_otg_20:25:402
215062284,15sleep10-21swapper/119:10:251
2177899271,8cyclictest7850irq/56-dwc_otg20:50:251
210502275,11sleep221144-21users19:05:532
2177899264,6cyclictest29701-21irqstats19:30:441
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional