You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 15:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Dec 28, 2025 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5123991001,37cyclictest0-21swapper/007:20:250
5123998514,57cyclictest15068-21ntpq09:05:320
5123998514,57cyclictest15068-21ntpq09:05:320
5123998116,37cyclictest30535-21munin-run09:50:090
5123998113,49cyclictest506-21fschecks_time09:55:270
5123998014,55cyclictest21169-21tr12:20:320
5123998014,52cyclictest3138-21unixbench_multi10:00:400
5123998014,52cyclictest3138-21unixbench_multi10:00:400
5123997918,46cyclictest11067-21cron11:55:090
5123997917,29cyclictest0-21swapper/007:30:230
5123997914,55cyclictest8772-21basename08:50:090
5123997811,51cyclictest13595-21latency10:30:290
5123997712,52cyclictest22320-21fschecks_count10:55:260
5123997612,49cyclictest0-21swapper/009:31:520
5123997513,39cyclictest30422-21grep11:15:370
5123997420,42cyclictest12898-21fschecks_time09:00:270
5123997315,43cyclictest9321-21perl08:50:260
5123997215,45cyclictest30860-21aten_rbpower_po09:50:230
5123997115,34cyclictest0-21swapper/008:55:190
5123997018,42cyclictest32119-21taskset08:24:350
5123997012,50cyclictest0-21swapper/007:59:180
5123997011,45cyclictest0-21swapper/011:35:200
308922700,1sleep10-21swapper/109:50:241
512399691,32cyclictest0-21swapper/010:40:220
5123996910,41cyclictest0-21swapper/010:25:210
5123996816,39cyclictest26574-21seq08:09:240
5126996732,18cyclictest7259-21wget08:45:233
5123996720,35cyclictest12246-21sensors11:55:340
5126996633,13cyclictest9583-21sed08:50:303
512399662,34cyclictest0-21swapper/008:35:220
5123996618,12cyclictest29076-21munin-run11:15:100
512499656,10cyclictest7750irq/56-dwc_otg07:15:211
5123996512,39cyclictest0-21swapper/012:34:490
5126996434,11cyclictest25386-21sshd12:31:373
512399649,41cyclictest0-21swapper/010:14:000
512399648,24cyclictest0-21swapper/009:25:230
512399648,24cyclictest0-21swapper/009:25:230
5123996418,25cyclictest27201-21fschecks_time08:10:260
5123996410,43cyclictest0-21swapper/011:21:110
5126996330,12cyclictest20201-21sed10:50:103
512399631,43cyclictest0-21swapper/012:15:280
5126996230,11cyclictest21173-21chrt12:20:323
5126996228,17cyclictest32656-21/usr/sbin/munin09:55:243
512399621,30cyclictest0-21swapper/007:25:210
512399621,30cyclictest0-21swapper/007:25:210
5123996211,44cyclictest0-21swapper/012:00:540
5123996211,43cyclictest16312-21taskset10:38:460
5123996211,38cyclictest0-21swapper/012:06:270
5126996132,7cyclictest11367-21df10:25:233
512399619,37cyclictest0-21swapper/008:17:190
5123996136,14cyclictest8869-21grep11:45:350
512399613,30cyclictest26490-21missed_timers12:35:330
5123996113,38cyclictest19355-21taskset07:49:180
5123996113,30cyclictest0-21swapper/007:51:470
5123996112,46cyclictest28861-21latency_hist09:45:100
5123996111,39cyclictest0-21swapper/010:50:100
5123996110,43cyclictest7172-21taskset11:41:240
5126996034,15cyclictest19379-21sshd07:49:373
512399609,42cyclictest0-21swapper/009:15:540
512399609,39cyclictest0-21swapper/007:37:080
512399607,42cyclictest0-21swapper/010:50:100
512399607,36cyclictest0-21swapper/007:40:240
5123996016,30cyclictest0-21swapper/007:10:120
5123996016,30cyclictest0-21swapper/007:10:120
5123996012,36cyclictest0-21swapper/011:09:350
5126995930,12cyclictest28936-21cat09:45:103
5126995929,17cyclictest15827-21taskset09:08:313
5126995929,17cyclictest15827-21taskset09:08:313
5126995929,13cyclictest11327-21dpkg11:55:103
512499595,10cyclictest7750irq/56-dwc_otg11:25:221
5124995915,10cyclictest0-21swapper/107:20:251
512399595,5cyclictest0-21swapper/007:15:230
512399592,30cyclictest1130-21unixbench_singl08:25:390
5123995911,38cyclictest25358-21chrt11:04:010
5123995910,46cyclictest3531-21sed08:35:100
512399584,47cyclictest0-21swapper/010:19:360
5123995814,33cyclictest0-21swapper/008:42:290
5123995811,36cyclictest20338-21if_err_wlan009:20:290
5126995731,12cyclictest1307-21taskset08:27:303
5126995729,9cyclictest0-21swapper/309:25:203
5126995729,9cyclictest0-21swapper/309:25:203
5126995727,14cyclictest5144-21kworker/u8:011:55:163
5126995723,23cyclictest0-21swapper/307:54:443
5126995718,25cyclictest7950irq/56-dwc_otg_08:40:223
512399578,40cyclictest18415-21runrttasks12:14:570
512399578,29cyclictest0-21swapper/011:25:270
512399577,34cyclictest0-21swapper/009:36:490
512399576,41cyclictest0-21swapper/009:10:330
5123995712,24cyclictest10458-21grep10:20:380
5126995630,7cyclictest4110-21apt-get11:35:213
5123995613,36cyclictest24607-21runrttasks08:02:140
5123995611,36cyclictest23541-21taskset12:26:080
5126995540,11cyclictest29594-21ls09:45:293
512399556,40cyclictest5266-21runrttasks10:08:330
5126995428,12cyclictest32568-21seq11:23:453
5126995426,12cyclictest0-21swapper/308:22:293
5126995333,8cyclictest26256-21ls11:05:323
5126995325,15cyclictest27462-21chrt12:40:003
5126995323,10cyclictest7950irq/56-dwc_otg_09:50:353
5126995231,10cyclictest5051-21chrt10:05:563
5126995229,12cyclictest16126-21chrt10:36:233
5126995226,14cyclictest19454-21rm09:18:313
5126995224,11cyclictest3631-21chrt11:32:263
5126995222,10cyclictest7950irq/56-dwc_otg_11:15:243
512399528,35cyclictest3664-21taskset11:32:500
5126995139,9cyclictest21487-21apt-get07:55:213
5126995126,16cyclictest7175-21chrt11:41:273
5126995126,11cyclictest1125-21runrttasks09:24:063
5126995125,15cyclictest26605-21runrttasks08:09:473
5126995120,14cyclictest7950irq/56-dwc_otg_10:17:153
5126995036,11cyclictest27342-21/usr/sbin/munin08:10:283
5126995036,11cyclictest15357-21/usr/sbin/munin12:05:273
5126995035,12cyclictest13837-21processes10:30:343
5126995029,11cyclictest12847-21fschecks_count09:00:273
5126995027,12cyclictest15553-21chrt07:36:463
5126995026,11cyclictest3299-21chrt08:34:513
5126994934,12cyclictest9559-21awk10:20:253
5126994924,7cyclictest17576-21sshd07:44:373
5126994923,11cyclictest3559-21chrt08:35:113
512499492,9cyclictest7750irq/56-dwc_otg10:15:231
5126994836,10cyclictest30061-21sshd08:17:373
5126994835,10cyclictest5908-21/usr/sbin/munin07:10:353
5126994835,10cyclictest5908-21/usr/sbin/munin07:10:353
5126994834,11cyclictest28206-21/usr/sbin/munin11:10:333
5126994834,11cyclictest20728-21/usr/sbin/munin10:50:303
5126994834,11cyclictest17488-21sed12:10:313
5126994834,10cyclictest13626-21grep07:30:373
512499488,26cyclictest0-21swapper/109:20:141
512699478,27cyclictest0-21swapper/307:15:153
5126994737,7cyclictest22314-21apt-get12:25:233
5126994735,10cyclictest746-21/usr/sbin/munin11:25:273
5126994734,10cyclictest23712-21/usr/sbin/munin08:00:303
5126994732,12cyclictest24473-21threads09:30:373
5126994723,11cyclictest22076-21/usr/sbin/munin10:55:293
512499473,8cyclictest7750irq/56-dwc_otg08:40:211
5126994633,10cyclictest11295-21ls07:25:303
5126994633,10cyclictest11295-21ls07:25:303
5126994633,10cyclictest10139-21sshd07:21:373
5126994632,11cyclictest7410-21sed10:15:103
5126994631,12cyclictest17232-21ls10:40:303
5126994622,9cyclictest0-21swapper/308:55:183
5126994622,8cyclictest0-21swapper/310:01:043
5126994622,8cyclictest0-21swapper/310:01:043
512599467,30cyclictest0-21swapper/207:40:222
512599467,24cyclictest0-21swapper/210:30:232
512599465,15cyclictest7850irq/56-dwc_otg_10:00:232
512599465,15cyclictest7850irq/56-dwc_otg_10:00:232
512499468,23cyclictest0-21swapper/110:35:251
512499467,26cyclictest0-21swapper/109:30:241
512499467,24cyclictest0-21swapper/110:20:151
512699458,24cyclictest0-21swapper/311:45:213
5126994533,10cyclictest26632-21sshd09:36:373
5126994533,10cyclictest19371-21ntp_offset12:15:343
512599452,6cyclictest0-21swapper/210:10:222
5125994516,8cyclictest0-21swapper/210:20:162
512499457,23cyclictest0-21swapper/112:00:261
46532452,36sleep00-21swapper/007:07:360
5126994433,8cyclictest23896-21apt-get11:00:233
5126994430,11cyclictest16445-21cat09:10:253
512599447,25cyclictest0-21swapper/207:25:222
512599447,25cyclictest0-21swapper/207:25:222
512599447,22cyclictest0-21swapper/209:10:222
512499447,24cyclictest0-21swapper/108:20:141
512499447,22cyclictest0-21swapper/110:05:231
512499444,9cyclictest0-21swapper/107:10:121
512499444,9cyclictest0-21swapper/107:10:121
512499442,6cyclictest7950irq/56-dwc_otg_08:45:211
512599437,22cyclictest0-21swapper/207:45:232
512599437,21cyclictest0-21swapper/209:55:152
5125994320,7cyclictest25683-21idleruntime-cro12:35:102
512499437,28cyclictest0-21swapper/111:35:221
512499437,24cyclictest0-21swapper/111:55:151
512499437,23cyclictest0-21swapper/112:35:261
512499435,2cyclictest0-21swapper/112:05:251
141482431,36sleep214168-21idleruntime-cro09:05:092
5126994230,10cyclictest13821-21/usr/sbin/munin12:00:333
512599422,8cyclictest7750irq/56-dwc_otg08:45:202
512499427,23cyclictest0-21swapper/110:55:231
512499427,22cyclictest0-21swapper/108:00:251
512499424,3cyclictest0-21swapper/111:05:251
512499422,10cyclictest0-21swapper/108:25:211
512599417,24cyclictest0-21swapper/209:05:242
512599417,24cyclictest0-21swapper/209:05:242
512599417,22cyclictest0-21swapper/210:40:232
5125994114,8cyclictest128-21systemd-journal09:29:272
5125994114,8cyclictest128-21systemd-journal09:29:272
5124994018,5cyclictest19839-21unixbench_singl12:15:401
44522400,5sleep2512599cyclictest11:35:272
512599397,25cyclictest0-21swapper/210:45:242
512599395,8cyclictest7950irq/56-dwc_otg_10:25:192
512499397,25cyclictest0-21swapper/111:50:221
512599385,8cyclictest0-21swapper/207:55:212
76022371,2sleep20-21swapper/210:15:252
512599379,5cyclictest7750irq/56-dwc_otg09:15:372
512599377,26cyclictest0-21swapper/208:55:222
512599371,31cyclictest4181-21ls10:05:272
512599371,21cyclictest20411-21ls12:20:102
512499374,4cyclictest7850irq/56-dwc_otg_11:44:561
512499373,6cyclictest23-21ksoftirqd/109:55:381
512499373,17cyclictest7950irq/56-dwc_otg_10:45:091
5124993720,6cyclictest27508-21/usr/sbin/munin11:10:321
512599363,8cyclictest0-21swapper/207:50:202
512599362,27cyclictest7950irq/56-dwc_otg_11:50:202
512599361,4cyclictest0-21swapper/209:50:222
5125993612,6cyclictest29-21rcuc/211:30:362
512499366,9cyclictest0-21swapper/108:15:221
512499366,7cyclictest7850irq/56-dwc_otg_11:00:291
512499361,25cyclictest20197-21latency07:50:291
36032364,17sleep33692-21/usr/sbin/munin07:05:293
512599356,5cyclictest7950irq/56-dwc_otg_10:50:242
512599356,18cyclictest0-21swapper/211:05:402
512599353,3cyclictest0-21swapper/209:45:222
5125993512,6cyclictest724-21cut11:25:272
512499356,6cyclictest7850irq/56-dwc_otg_08:30:391
512499356,5cyclictest14717-21fschecks_time09:05:281
512499356,5cyclictest14717-21fschecks_time09:05:281
512499354,20cyclictest0-21swapper/110:25:191
5124993515,6cyclictest0-21swapper/108:50:201
512499351,4cyclictest0-21swapper/111:35:091
512499351,24cyclictest17824-21timerandwakeup12:10:361
33122355,20sleep23308-21apt-get07:05:242
512599343,19cyclictest29226-21grep08:15:302
512499346,5cyclictest7850irq/56-dwc_otg_12:20:241
512499346,15cyclictest0-21swapper/109:05:101
512499342,6cyclictest8188-21seq11:45:271
512499341,9cyclictest17647-21munin-plugin-st07:45:091
5124993414,6cyclictest28856-21latency_hist09:45:101
512499341,24cyclictest0-21swapper/109:35:291
274612340,4sleep10-21swapper/108:10:291
512599333,22cyclictest0-21swapper/209:38:072
512599333,14cyclictest7950irq/56-dwc_otg_08:40:282
512599332,6cyclictest5295-21latency_hist08:40:102
512599332,19cyclictest0-21swapper/212:05:242
512599331,23cyclictest31034-21ls11:20:092
512599331,15cyclictest7750irq/56-dwc_otg12:20:322
512499337,4cyclictest0-21swapper/110:10:341
512499332,21cyclictest0-21swapper/108:38:181
512499331,9cyclictest15373-21unixbench_multi07:35:401
5124993314,6cyclictest2358-21fschecks_time10:00:291
5124993314,6cyclictest2358-21fschecks_time10:00:291
5124993314,14cyclictest23-21ksoftirqd/112:25:231
35122335,16sleep13523-21diskstats07:05:271
512599326,6cyclictest29-21rcuc/210:35:322
5125993218,5cyclictest6903-21timerwakeupswit11:40:372
512599321,5cyclictest7750irq/56-dwc_otg07:35:292
512599321,24cyclictest22283-21ls10:55:242
512599321,18cyclictest32694-21latency_hist11:25:102
512599321,11cyclictest143142unixbench_multi12:00:402
512499327,13cyclictest7950irq/56-dwc_otg_09:15:361
512499323,7cyclictest7950irq/56-dwc_otg_10:55:091
512499321,4cyclictest0-21swapper/107:35:101
512599317,6cyclictest7750irq/56-dwc_otg07:20:362
512599314,6cyclictest7950irq/56-dwc_otg_08:50:232
512599314,6cyclictest2040-21fschecks_count08:30:282
512599312,14cyclictest0-21swapper/209:20:392
512599311,12cyclictest28453-21cron08:15:102
5125993110,6cyclictest18026-21unixbench_singl12:10:382
512499316,11cyclictest0-21swapper/110:45:291
512499313,5cyclictest0-21swapper/108:05:221
512499312,6cyclictest0-21swapper/107:50:101
512499312,3cyclictest0-21swapper/111:20:091
512499311,4cyclictest7850irq/56-dwc_otg_09:45:351
512599307,20cyclictest0-21swapper/212:25:252
512599306,14cyclictest0-21swapper/207:15:162
512599305,5cyclictest7850irq/56-dwc_otg_07:10:322
512599305,5cyclictest7850irq/56-dwc_otg_07:10:322
512599302,10cyclictest0-21swapper/208:00:232
512599301,7cyclictest8385-21ntpq11:45:292
512599301,20cyclictest0-21swapper/208:28:442
512599301,11cyclictest7850irq/56-dwc_otg_08:20:352
5125993010,12cyclictest26418-21latency12:35:322
512499307,5cyclictest7950irq/56-dwc_otg_10:30:261
512499304,9cyclictest0-21swapper/112:30:241
512499304,12cyclictest7950irq/56-dwc_otg_11:25:091
512599296,13cyclictest7950irq/56-dwc_otg_08:10:102
512599292,7cyclictest7750irq/56-dwc_otg11:10:272
5125992923,4cyclictest27194-21aten_rbpower_cu09:40:232
512599292,16cyclictest0-21swapper/211:00:482
512599291,6cyclictest11518-21aten_rbpower_po11:55:232
512499294,13cyclictest7850irq/56-dwc_otg_07:25:211
512499294,13cyclictest7850irq/56-dwc_otg_07:25:211
512499293,6cyclictest1125-21runrttasks09:27:151
512499293,6cyclictest1125-21runrttasks09:27:151
5124992911,5cyclictest23-21ksoftirqd/109:10:361
512599281,19cyclictest0-21swapper/207:31:482
512599281,17cyclictest0-21swapper/209:30:332
512499289,5cyclictest11221-21awk08:55:271
512499277,9cyclictest7950irq/56-dwc_otg_07:55:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional