You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 11:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Jan 11, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
736821490,10sleep27370-21timerwakeupswit19:05:452
81999910220,45cyclictest0-21swapper/019:50:290
8199999830,26cyclictest0-21swapper/023:35:290
819999892,9cyclictest128-21systemd-journal21:10:160
819999891,7cyclictest14593-21fschecks_count22:25:340
311272880,5sleep3820299cyclictest21:40:413
105482860,3sleep30-21swapper/319:15:343
819999841,68cyclictest25883-21missed_timers22:55:370
284102840,2sleep00-21swapper/020:05:310
819999822,6cyclictest0-21swapper/019:35:260
819999812,6cyclictest5035-21aten_rbpower_po20:30:300
8199998125,43cyclictest7950irq/56-dwc_otg_21:10:380
819999801,61cyclictest26519-21/usr/sbin/munin20:00:440
819999792,5cyclictest0-21swapper/021:55:440
819999791,59cyclictest0-21swapper/020:20:160
8199997915,38cyclictest22023-21grep19:45:450
819999781,63cyclictest0-21swapper/022:50:290
819999772,53cyclictest0-21swapper/021:05:170
819999771,10cyclictest32343-21date23:15:160
819999761,61cyclictest13049-21latency22:20:340
119172740,1sleep20-21swapper/222:15:462
819999731,9cyclictest24207-21head21:20:470
321232730,2sleep332126-21cpuspeed20:15:323
819999725,35cyclictest7850irq/56-dwc_otg_21:45:290
819999722,7cyclictest2639-21grep21:50:450
8199997218,40cyclictest7950irq/56-dwc_otg_20:57:230
819999721,54cyclictest20026-21sort00:10:170
275312720,1sleep30-21swapper/321:30:433
819999711,56cyclictest8381-21chrt22:06:380
8199997115,31cyclictest0-21swapper/023:45:170
819999702,8cyclictest0-21swapper/000:15:410
819999702,8cyclictest0-21swapper/000:15:410
819999692,6cyclictest0-21swapper/022:44:410
8199996921,30cyclictest2910-21awk20:25:160
819999682,9cyclictest0-21swapper/023:56:230
819999682,6cyclictest3025-21unixbench_multi23:20:460
819999682,6cyclictest0-21swapper/019:30:160
819999681,54cyclictest3130-21/usr/sbin/munin20:25:340
819999681,51cyclictest21221-21/usr/sbin/munin21:15:350
8199996717,39cyclictest20840-21open_files00:10:390
819999671,54cyclictest0-21swapper/020:41:080
819999671,3cyclictest0-21swapper/020:35:440
8199996712,44cyclictest30557-21seq00:38:090
8199996712,41cyclictest104442chrt22:14:470
8199996710,46cyclictest0-21swapper/023:31:360
819999669,3cyclictest7950irq/56-dwc_otg_21:40:260
819999662,7cyclictest0-21swapper/019:11:370
819999661,9cyclictest0-21swapper/000:32:110
819999661,8cyclictest0-21swapper/021:36:540
819999661,8cyclictest0-21swapper/021:36:540
8199996615,37cyclictest0-21swapper/019:40:290
8199996614,44cyclictest0-21swapper/019:55:310
8199996614,44cyclictest0-21swapper/019:55:310
8199996612,41cyclictest0-21swapper/023:45:270
820099648,5cyclictest32115-21runrttasks21:45:271
8199996424,28cyclictest0-21swapper/023:00:300
819999641,55cyclictest0-21swapper/021:28:540
820099634,4cyclictest7850irq/56-dwc_otg_22:55:411
819999631,51cyclictest0-21swapper/020:15:000
819999621,4cyclictest26716-21aten_rbpower_cu21:30:300
819999621,45cyclictest3842-21grep23:25:300
820099617,4cyclictest3130-21/usr/sbin/munin20:25:361
820099612,7cyclictest11069-21apt-get23:45:271
819999611,5cyclictest32578-21aten_rbpower_cu23:15:300
819999611,45cyclictest0-21swapper/023:07:010
819999611,45cyclictest0-21swapper/023:07:010
8199996112,38cyclictest118412sleep020:49:060
8199996112,38cyclictest118412sleep020:49:060
8199996110,43cyclictest174232sleep022:31:300
8199996110,43cyclictest174232sleep022:31:300
8202996026,13cyclictest7677-21/usr/sbin/munin20:35:433
8202996021,30cyclictest20280-21uptime19:40:453
819999608,38cyclictest0-21swapper/022:36:350
8199996016,33cyclictest0-21swapper/019:30:190
8199996011,38cyclictest180462sleep000:05:080
8202995922,9cyclictest24386-21munin-run19:55:173
819999597,44cyclictest0-21swapper/022:17:300
819999597,29cyclictest12093-21fschecks_count19:20:320
819999593,46cyclictest0-21swapper/022:03:580
8202995817,26cyclictest20804-21ntp_offset00:10:383
820199583,12cyclictest7850irq/56-dwc_otg_22:20:262
820199581,9cyclictest0-21swapper/200:10:282
820099589,10cyclictest26645-21taskset21:30:181
820099583,8cyclictest0-21swapper/100:35:281
820099582,5cyclictest5436-21cut20:30:361
820099581,5cyclictest7850irq/56-dwc_otg_21:50:331
819999581,44cyclictest24997-21taskset00:21:180
8202995725,20cyclictest128-21systemd-journal19:35:163
8202995724,13cyclictest22632-21sshd21:18:063
820099573,6cyclictest13146-21sed22:20:351
820099571,6cyclictest2240-21fschecks_time23:20:341
819999578,34cyclictest23055-21taskset22:47:480
819999571,45cyclictest0-21swapper/020:51:440
819999571,45cyclictest0-21swapper/020:51:440
8199995711,39cyclictest268412sleep000:26:500
102872570,18sleep00-21swapper/019:15:310
8202995617,21cyclictest7950irq/56-dwc_otg_22:25:463
820099563,5cyclictest0-21swapper/119:25:271
820099561,16cyclictest0-21swapper/121:32:001
8202995524,21cyclictest18951-21sshd21:07:373
8202995524,13cyclictest25571-21/usr/sbin/munin22:55:333
820199554,15cyclictest0-21swapper/221:55:272
820099552,5cyclictest20920-21ps00:10:401
820099552,4cyclictest17031-21modprobe19:35:161
820099551,6cyclictest21912-21ntpq21:15:381
820099551,5cyclictest23750-21fschecks_count22:50:331
8200995515,5cyclictest1125-21runrttasks21:35:391
8200995515,5cyclictest1125-21runrttasks21:35:391
820099551,3cyclictest0-21swapper/121:10:371
8200995513,4cyclictest18045-21if_wlan021:05:361
8202995425,12cyclictest21569-21/usr/sbin/munin19:45:393
820099544,5cyclictest15426-21grep22:25:451
820099544,4cyclictest128-21systemd-journal22:20:171
820099541,6cyclictest0-21swapper/119:45:441
820099541,5cyclictest9006-21latency_hist23:40:171
8202995324,14cyclictest19894-21awk00:10:173
8202995319,19cyclictest7850irq/56-dwc_otg_19:35:323
820099534,8cyclictest7850irq/56-dwc_otg_19:40:501
820099532,10cyclictest0-21swapper/100:25:311
820099531,3cyclictest0-21swapper/119:35:351
8200995313,4cyclictest7045-21grep20:35:331
8200995312,4cyclictest16246-21aten_rbpower_po22:30:291
8200995312,4cyclictest16246-21aten_rbpower_po22:30:291
8202995226,12cyclictest3203-21/usr/sbin/munin20:25:313
8202995217,19cyclictest20120-21ntpq21:10:383
820099529,39cyclictest0-21swapper/122:45:461
820099521,3cyclictest0-21swapper/119:15:371
820099521,2cyclictest0-21swapper/122:00:451
8202995122,11cyclictest1460-21/usr/sbin/munin21:50:423
8202995121,17cyclictest32456-21cat23:15:173
8202995120,13cyclictest1934-21cut23:20:303
8202995117,16cyclictest13372-21taskset19:24:433
8202995117,12cyclictest23517-21/usr/sbin/munin22:50:293
820099512,3cyclictest0-21swapper/119:50:421
8202995031,7cyclictest1338-21apt-get20:20:283
8202995022,13cyclictest13663-21taskset20:54:153
8202995022,13cyclictest13663-21taskset20:54:153
8202995017,16cyclictest10910-21cut23:45:173
820099501,5cyclictest0-21swapper/100:00:301
820099493,9cyclictest9450irq/48-DMA10:44:551
820099491,6cyclictest1125-21runrttasks23:17:471
820099491,3cyclictest7850irq/56-dwc_otg_23:34:251
8199994911,35cyclictest143002chrt23:53:330
820299487,26cyclictest0-21swapper/323:00:293
8202994823,12cyclictest29180-21apt00:35:283
8202994815,11cyclictest0-21swapper/322:01:113
8202994814,20cyclictest7950irq/56-dwc_otg_22:20:363
820199482,8cyclictest7750irq/56-dwc_otg19:25:272
820099487,4cyclictest11865-21seq20:49:261
820099487,4cyclictest11865-21seq20:49:261
820099482,4cyclictest0-21swapper/121:05:021
820099481,3cyclictest0-21swapper/123:05:331
820099481,3cyclictest0-21swapper/123:05:331
8202994732,13cyclictest18475-21latency22:35:423
8202994724,11cyclictest14307-21sshd23:53:383
8202994720,12cyclictest5029-21aten_rbpower_po20:30:303
8202994719,13cyclictest22657-21proc_pri00:15:413
8202994719,13cyclictest22657-21proc_pri00:15:413
8202994718,8cyclictest4358-21ls23:25:373
8202994716,13cyclictest10009-21taskset20:43:473
8202994713,23cyclictest7950irq/56-dwc_otg_23:09:263
8202994713,23cyclictest7950irq/56-dwc_otg_23:09:263
820099474,5cyclictest0-21swapper/120:54:251
820099474,5cyclictest0-21swapper/120:54:251
820099474,3cyclictest7950irq/56-dwc_otg_22:39:421
820099472,7cyclictest0-21swapper/100:21:221
820099471,9cyclictest0-21swapper/122:09:241
8200994716,26cyclictest0-21swapper/119:55:231
8200994716,26cyclictest0-21swapper/119:55:231
8200994715,4cyclictest4854-21idleruntime-cro22:00:161
8202994620,12cyclictest15240-21awk19:30:163
8202994615,20cyclictest15529-21seq21:00:013
8202994613,12cyclictest12150-21/usr/sbin/munin23:45:443
820199467,24cyclictest0-21swapper/222:35:362
820099467,31cyclictest0-21swapper/120:20:281
820099462,3cyclictest7750irq/56-dwc_otg20:55:261
820099461,6cyclictest15586-21switchtime23:55:421
820099461,4cyclictest23422-21idleruntime21:20:351
8200994614,5cyclictest1125-21runrttasks19:10:351
820299457,25cyclictest0-21swapper/319:10:293
8202994519,11cyclictest1124-21snmpd19:56:313
8202994519,11cyclictest1124-21snmpd19:56:313
8202994517,11cyclictest17614-21taskset22:33:503
8202994517,11cyclictest17614-21taskset22:33:503
8202994515,14cyclictest29852-21chrt20:09:443
8202994514,12cyclictest7950irq/56-dwc_otg_21:45:273
820199457,23cyclictest0-21swapper/219:40:302
820199453,6cyclictest0-21swapper/220:20:282
820099451,9cyclictest28024-21runrttasks20:04:321
820099451,4cyclictest7750irq/56-dwc_otg00:15:321
820099451,4cyclictest7750irq/56-dwc_otg00:15:321
8200994513,4cyclictest9949-21ntp_kernel_pll_23:40:401
820099451,2cyclictest0-21swapper/100:05:311
820299447,24cyclictest0-21swapper/323:30:303
8202994422,11cyclictest486-21cat23:15:353
8202994417,10cyclictest8198-21cyclictest21:55:453
8202994416,11cyclictest7850irq/56-dwc_otg_22:42:003
8201994415,6cyclictest21299-21aten_rbpower_po21:15:292
820099441,5cyclictest1125-21runrttasks23:12:361
820099441,3cyclictest0-21swapper/122:10:411
8202994332,8cyclictest21679-21apt-get22:45:223
8202994317,12cyclictest1124-21snmpd20:46:423
8202994317,12cyclictest1124-21snmpd20:46:423
8202994315,13cyclictest17112-21chrt21:02:133
8202994315,13cyclictest128-21systemd-journal20:01:333
8202994313,21cyclictest7950irq/56-dwc_otg_20:15:013
8202994312,15cyclictest7750irq/56-dwc_otg00:24:033
820199434,4cyclictest7750irq/56-dwc_otg19:50:262
8202994217,16cyclictest28906-21runrttasks00:35:033
820199426,24cyclictest0-21swapper/219:15:282
820099423,11cyclictest8585-21runrttasks20:40:281
820099421,5cyclictest29250-21ls20:05:431
820299417,22cyclictest0-21swapper/300:25:303
820299417,21cyclictest0-21swapper/321:25:313
8202994118,10cyclictest0-21swapper/323:39:533
8202994116,10cyclictest0-21swapper/322:09:203
8202994111,11cyclictest0-21swapper/321:23:313
820199417,21cyclictest0-21swapper/220:40:302
820199411,11cyclictest0-21swapper/223:20:292
80492410,2sleep10-21swapper/119:10:171
8202994029,8cyclictest28495-21apt-get21:35:293
8202994029,8cyclictest28495-21apt-get21:35:293
820199407,24cyclictest0-21swapper/221:30:302
820199403,11cyclictest7750irq/56-dwc_otg21:50:282
8200994016,5cyclictest5307-21idleruntime-cro23:30:161
8202993925,11cyclictest15005-21/usr/sbin/munin23:55:333
8202993913,12cyclictest10460-21taskset22:15:013
8202993912,9cyclictest0-21swapper/322:15:383
8201993910,6cyclictest7950irq/56-dwc_otg_21:35:342
8201993910,6cyclictest7950irq/56-dwc_otg_21:35:342
820199387,14cyclictest7950irq/56-dwc_otg_23:30:392
820199383,13cyclictest7850irq/56-dwc_otg_20:30:302
8200993817,5cyclictest20622-21/usr/sbin/munin22:40:441
8202993724,11cyclictest16903-21/usr/sbin/munin00:00:353
820199378,7cyclictest7950irq/56-dwc_otg_22:55:172
8201993715,5cyclictest29216-21aten_rbpower_en00:35:312
820099365,4cyclictest7850irq/56-dwc_otg_00:30:441
8200993611,5cyclictest21-21rcuc/120:10:461
820199357,16cyclictest0-21swapper/222:55:292
820199357,12cyclictest7950irq/56-dwc_otg_23:35:382
8201993529,4cyclictest25732-21ls21:25:432
820199351,5cyclictest7950irq/56-dwc_otg_22:45:292
820199351,24cyclictest30471-21ls21:40:302
8200993516,4cyclictest12934-21aten_rbpower_en23:50:301
820199346,7cyclictest7850irq/56-dwc_otg_19:20:432
820199345,7cyclictest7850irq/56-dwc_otg_23:15:422
820199342,6cyclictest16754-21fschecks_count00:00:332
820099342,3cyclictest0-21swapper/119:20:261
64452340,2sleep06448-21aten_rbpower_po19:05:310
820199337,5cyclictest7950irq/56-dwc_otg_22:05:452
820199333,17cyclictest7950irq/56-dwc_otg_22:10:372
820199333,14cyclictest32397-21fschecks_time21:45:342
8201993327,4cyclictest301ktimersoftd/220:05:282
820199332,11cyclictest7850irq/56-dwc_otg_19:45:412
820199328,21cyclictest0-21swapper/223:55:292
820199327,11cyclictest7950irq/56-dwc_otg_20:55:382
820199326,5cyclictest17260-21tail22:30:442
820199326,5cyclictest17260-21tail22:30:442
820199322,16cyclictest0-21swapper/221:25:172
820199321,22cyclictest29463-21ls23:05:382
820199321,22cyclictest29463-21ls23:05:382
8201993210,6cyclictest0-21swapper/219:15:172
820199319,13cyclictest7950irq/56-dwc_otg_23:50:352
820199317,21cyclictest0-21swapper/222:00:222
820199316,13cyclictest7950irq/56-dwc_otg_23:25:402
820199315,10cyclictest7950irq/56-dwc_otg_21:10:302
820199314,22cyclictest0-21swapper/200:05:312
820199313,18cyclictest0-21swapper/222:41:372
8201993112,12cyclictest13494-21taskset20:52:122
8201993112,12cyclictest13494-21taskset20:52:122
820199311,18cyclictest27426-21ls20:00:442
820199311,18cyclictest0-21swapper/220:19:352
820199311,17cyclictest24640-21switchtime00:20:432
820099316,10cyclictest0-21swapper/121:40:371
820099315,5cyclictest7750irq/56-dwc_otg23:00:401
76482313,21sleep30-21swapper/319:06:443
820199307,5cyclictest7750irq/56-dwc_otg00:30:162
820199305,17cyclictest7750irq/56-dwc_otg23:45:302
820199304,5cyclictest7850irq/56-dwc_otg_20:10:372
820199303,12cyclictest7850irq/56-dwc_otg_23:00:452
820199302,7cyclictest7950irq/56-dwc_otg_20:30:162
820199302,15cyclictest14984-21cat22:25:382
820199301,13cyclictest7850irq/56-dwc_otg_23:10:302
820199297,5cyclictest7950irq/56-dwc_otg_21:05:422
820199297,20cyclictest0-21swapper/200:15:292
820199297,20cyclictest0-21swapper/200:15:292
820199292,6cyclictest128-21systemd-journal21:01:452
820199292,13cyclictest7850irq/56-dwc_otg_00:30:372
8201992910,5cyclictest29-21rcuc/223:40:402
820199283,4cyclictest0-21swapper/219:30:402
820199275,5cyclictest7950irq/56-dwc_otg_19:55:232
820199275,5cyclictest7950irq/56-dwc_otg_19:55:232
820199272,5cyclictest7750irq/56-dwc_otg20:40:172
820199261,4cyclictest10603-21diskstats20:45:322
820199261,4cyclictest10603-21diskstats20:45:322
820199261,15cyclictest0-21swapper/219:35:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional