You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 13:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Jul 26, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3251821690,2sleep31336999cyclictest22:37:433
13367991193,84cyclictest7950irq/56-dwc_otg_00:20:491
133669911733,55cyclictest9732-21ssh21:30:510
133669911733,55cyclictest9732-21ssh21:30:510
133689911544,33cyclictest8791-21/usr/sbin/munin21:30:522
133689911544,33cyclictest8791-21/usr/sbin/munin21:30:522
133689911436,56cyclictest125-21systemd-journal23:01:042
133669911322,85cyclictest101ktimersoftd/022:30:580
133669911123,9cyclictest101ktimersoftd/021:35:530
13369991103,77cyclictest0-21swapper/300:20:503
13367991103,74cyclictest4926-21ssh21:25:401
13367991102,80cyclictest19880-21ssh23:01:061
133669910855,31cyclictest10946-21grep22:50:490
13367991048,47cyclictest28085-21ssh23:50:451
13366991047,56cyclictest0-21swapper/021:20:480
13366991042,79cyclictest8050irq/56-dwc_otg_23:55:540
133689910234,16cyclictest7930-21irqstats00:05:482
133689910026,17cyclictest28243-21ntp_states23:50:502
1336699995,59cyclictest0-21swapper/000:30:530
1336999985,23cyclictest0-21swapper/323:50:263
13368999746,16cyclictest20579-21ntp_states00:20:472
1336799973,62cyclictest10270-21ntp_states22:10:491
1336699973,68cyclictest0-21swapper/022:25:540
1336699973,68cyclictest0-21swapper/022:25:540
13366999714,44cyclictest0-21swapper/000:05:390
1336999963,8cyclictest7850irq/56-dwc_otg21:30:533
1336999963,8cyclictest7850irq/56-dwc_otg21:30:533
13366999648,42cyclictest0-21swapper/019:50:290
1336699963,72cyclictest0-21swapper/023:50:280
1336699953,50cyclictest18429-21ssh23:00:340
1336699935,64cyclictest4935-21df21:25:400
1336799929,62cyclictest14296-21apt-get22:55:371
1336799929,62cyclictest14296-21apt-get22:55:371
13366999257,17cyclictest0-21swapper/022:55:350
13366999257,17cyclictest0-21swapper/022:55:350
1336999913,12cyclictest27385-21ssh22:30:583
13368999128,36cyclictest0-21swapper/221:35:532
1336699914,58cyclictest20593-21ntp_states00:20:500
250422900,2sleep30-21swapper/322:29:053
250422900,2sleep30-21swapper/322:29:053
1336799903,69cyclictest9753-21/usr/sbin/munin21:30:521
1336799903,69cyclictest9753-21/usr/sbin/munin21:30:521
1336999893,19cyclictest8050irq/56-dwc_otg_21:25:433
13369998926,57cyclictest727-21runrttasks21:35:393
13368998938,24cyclictest27316-21ssh22:30:582
13368998935,23cyclictest32029-21ssh23:15:552
1336799899,74cyclictest26518-21ssh22:30:451
1336799894,64cyclictest0-21swapper/122:25:431
1336799894,64cyclictest0-21swapper/122:25:431
1336799893,80cyclictest24241-21diskmemload00:05:501
1336999883,19cyclictest32166-21ssh23:15:573
1336799882,79cyclictest317-21ssh23:17:041
1336799873,77cyclictest13185-21ssh21:35:421
13366998732,21cyclictest32123-21grep23:15:560
1336999863,73cyclictest0-21swapper/323:55:533
1336799863,76cyclictest1340-21ntp_states21:20:491
1336999853,15cyclictest10826-21missed_timers22:50:473
13368998527,20cyclictest0-21swapper/222:10:492
13366998532,33cyclictest0-21swapper/023:06:060
1336699852,44cyclictest24241-21diskmemload22:45:370
327572840,2sleep00-21swapper/022:38:080
1336999844,17cyclictest27691-21sort23:10:513
13366998423,33cyclictest27636-21smartpower23:10:500
13369998313,8cyclictest8050irq/56-dwc_otg_00:05:423
13368998340,25cyclictest32700-21timerwakeupswit23:55:532
1336799836,62cyclictest8050irq/56-dwc_otg_22:50:481
1336799833,58cyclictest0-21swapper/123:06:081
13368998232,19cyclictest4982-21ssh21:25:432
1336799812,71cyclictest28973-21/usr/sbin/munin00:30:511
1336699819,56cyclictest23928-21ssh00:25:370
13366998026,38cyclictest0-21swapper/022:10:510
13368997929,23cyclictest27600-21ssh23:10:502
1336799793,69cyclictest23936-21ssh00:25:391
1336999782,8cyclictest8050irq/56-dwc_otg_22:55:353
1336999782,8cyclictest8050irq/56-dwc_otg_22:55:353
13369997825,48cyclictest28986-21cat00:30:523
1336799785,55cyclictest7850irq/56-dwc_otg23:55:531
1336799782,69cyclictest27614-21/usr/sbin/munin23:10:501
1336999776,11cyclictest0-21swapper/321:20:493
1336999745,12cyclictest0-21swapper/322:10:503
1336899749,42cyclictest7950irq/56-dwc_otg_21:20:472
13368997443,25cyclictest29024-21sed00:30:532
1336999733,14cyclictest23913-21apt-get00:25:373
13368997345,22cyclictest23935-21ssh00:25:382
1336699735,20cyclictest7850irq/56-dwc_otg20:50:380
13366997348,5cyclictest8050irq/56-dwc_otg_22:40:350
13368997243,23cyclictest22719-21ls22:25:492
13368997243,23cyclictest22719-21ls22:25:492
1336999715,17cyclictest0-21swapper/323:01:053
13366997029,13cyclictest8050irq/56-dwc_otg_22:15:380
1336699699,28cyclictest0-21swapper/020:25:380
1336899685,11cyclictest7950irq/56-dwc_otg_22:50:472
1336699682,29cyclictest0-21swapper/020:15:350
1336999675,6cyclictest8050irq/56-dwc_otg_22:40:533
13368996623,13cyclictest32126-21ssh21:58:152
1336699642,28cyclictest61-21kswapd021:50:370
1336699641,44cyclictest3560-21fschecks_count00:00:430
1336999632,3cyclictest8050irq/56-dwc_otg_20:05:373
13368996335,22cyclictest14296-21apt-get22:55:352
13368996335,22cyclictest14296-21apt-get22:55:352
13368996317,28cyclictest8050irq/56-dwc_otg_22:40:472
1336799632,5cyclictest7850irq/56-dwc_otg23:20:491
1336799632,5cyclictest7850irq/56-dwc_otg23:20:491
1336999625,5cyclictest7950irq/56-dwc_otg_19:10:523
13368996223,24cyclictest125-21systemd-journal23:06:052
13368996222,21cyclictest32211-21ssh21:19:352
1336799628,38cyclictest0-21swapper/120:05:501
1336799621,52cyclictest0-21swapper/120:00:361
1336699622,40cyclictest32201-21ssh00:35:380
13366996213,22cyclictest13854-21fschecks_time20:40:410
1336999617,10cyclictest0-21swapper/323:36:343
1336799618,38cyclictest8050irq/56-dwc_otg_22:40:481
13367996110,37cyclictest0-21swapper/121:46:201
13366996117,25cyclictest9-21ksoftirqd/022:23:380
1336799605,45cyclictest8050irq/56-dwc_otg_23:29:291
13367996010,34cyclictest0-21swapper/122:00:481
1336699604,33cyclictest9-21ksoftirqd/021:00:420
13366996025,5cyclictest0-21swapper/023:20:360
13366996025,5cyclictest0-21swapper/023:20:360
117742600,5sleep20-21swapper/219:05:402
1336799596,39cyclictest30701-21chrt22:35:431
1336799596,38cyclictest0-21swapper/100:36:251
13366995910,31cyclictest9-21ksoftirqd/019:15:500
1336999586,12cyclictest8050irq/56-dwc_otg_20:55:393
1336999586,12cyclictest8050irq/56-dwc_otg_20:55:393
1336799581,5cyclictest3241-21/usr/sbin/munin00:00:511
1336799581,47cyclictest18301-21ssh00:18:181
1336799581,45cyclictest0-21swapper/123:32:441
1336799581,3cyclictest7850irq/56-dwc_otg21:55:491
1336699581,39cyclictest20187-21grep20:55:540
1336699581,39cyclictest20187-21grep20:55:540
1336699581,33cyclictest0-21swapper/023:35:370
13368995721,20cyclictest8050irq/56-dwc_otg_00:12:182
13368995718,25cyclictest22183-21rm23:44:092
1336799574,44cyclictest8050irq/56-dwc_otg_20:16:011
13368995620,20cyclictest2989-21tune2fs20:10:422
13368995620,20cyclictest14067-21awk19:10:462
1336799564,7cyclictest8050irq/56-dwc_otg_20:10:471
1336799562,10cyclictest0-21swapper/123:44:261
1336799561,6cyclictest7850irq/56-dwc_otg20:55:391
1336799561,6cyclictest7850irq/56-dwc_otg20:55:391
1336799561,6cyclictest0-21swapper/122:10:241
13366995624,15cyclictest21685-21grep19:30:510
1336699562,30cyclictest727-21runrttasks21:55:520
13366995612,24cyclictest9-21ksoftirqd/023:40:440
13368995523,19cyclictest27951-21ssh21:14:272
13368995517,23cyclictest32636-21ssh22:37:552
1336799559,31cyclictest0-21swapper/122:22:511
1336799557,4cyclictest8050irq/56-dwc_otg_22:18:161
1336799553,39cyclictest0-21swapper/121:45:241
13368995424,17cyclictest19803-21/usr/sbin/munin20:55:502
13368995424,17cyclictest19803-21/usr/sbin/munin20:55:502
13368995421,12cyclictest29251-21date21:55:242
1336799546,37cyclictest7950irq/56-dwc_otg_19:44:561
1336799546,37cyclictest7950irq/56-dwc_otg_19:44:561
1336699546,30cyclictest13-21rcuc/019:10:350
13366995429,5cyclictest7950irq/56-dwc_otg_21:40:530
13366995426,12cyclictest7850irq/56-dwc_otg23:25:480
1336699542,33cyclictest22521-21grep21:45:540
1336699541,6cyclictest0-21swapper/019:35:360
1336699541,30cyclictest24476-21grep23:45:530
1336699541,30cyclictest24476-21grep23:45:530
286432530,1sleep10-21swapper/121:54:281
13368995326,13cyclictest13739-21/usr/sbin/munin22:15:402
13368995324,12cyclictest17602-21ssh23:38:202
13368995317,14cyclictest21933-21grep21:00:532
1336799533,8cyclictest13646-21runrttasks00:12:221
1336799531,4cyclictest7950irq/56-dwc_otg_23:49:351
1336799531,4cyclictest7950irq/56-dwc_otg_23:49:351
1336799531,44cyclictest0-21swapper/119:10:501
13367995313,37cyclictest27595-21latency_hist19:50:241
13367995311,29cyclictest24241-21diskmemload21:16:501
1336699533,6cyclictest0-21swapper/022:05:380
1336699531,33cyclictest0-21swapper/021:15:340
13368995226,12cyclictest1182-21/usr/sbin/munin20:05:422
13368995220,21cyclictest26713-21ssh23:49:522
13368995220,21cyclictest26713-21ssh23:49:522
13368995216,14cyclictest4023-21tr00:00:482
1336799521,7cyclictest8219-21cut20:25:381
1336799521,7cyclictest0-21swapper/122:46:451
1336799521,4cyclictest7950irq/56-dwc_otg_19:20:441
1336799521,43cyclictest0-21swapper/123:35:391
1336799521,3cyclictest0-21swapper/120:35:411
13366995225,7cyclictest30868-21runrttasks19:55:540
13368995122,12cyclictest22967-21ssh21:46:392
13368995121,20cyclictest8686-21ssh22:09:352
13368995121,13cyclictest2633-21ssh22:01:052
13368995120,12cyclictest18265-21sh00:18:132
1336799512,7cyclictest750-21snmpd21:10:421
1336799511,7cyclictest18848-21wget19:25:381
1336799511,7cyclictest14605-21grep20:40:531
1336799511,6cyclictest6586-21grep20:20:421
1336799511,6cyclictest6586-21grep20:20:421
1336799511,6cyclictest20900-21aten_rbpower_cu21:00:381
13367995116,32cyclictest16906-21latency_hist20:50:241
1336699514,14cyclictest1504-21rm22:00:410
13366995119,22cyclictest0-21swapper/019:42:180
13366995119,22cyclictest0-21swapper/019:42:180
13368995024,13cyclictest18715-21ssh21:41:322
13368995017,22cyclictest31090-21chrt19:57:552
13368995016,13cyclictest10720-21taskset23:30:292
1336799501,11cyclictest0-21swapper/119:55:051
1336799501,10cyclictest0-21swapper/119:55:391
1336699506,25cyclictest11913-21ls20:35:400
13366995019,10cyclictest0-21swapper/020:00:370
13369994915,6cyclictest8050irq/56-dwc_otg_00:00:383
1336999491,20cyclictest0-21swapper/323:50:133
1336999491,20cyclictest0-21swapper/323:50:133
13369994911,19cyclictest8050irq/56-dwc_otg_23:06:053
13368994918,21cyclictest9429-21ssh22:49:412
13368994918,11cyclictest629-21ssh00:35:502
1336799492,14cyclictest0-21swapper/119:30:411
1336799491,7cyclictest22728-21aten_rbpower_en21:05:391
1336699499,8cyclictest7950irq/56-dwc_otg_19:20:350
1336699495,24cyclictest1650-21proc_pri20:05:490
13366994922,8cyclictest0-21swapper/020:45:470
1336699491,34cyclictest0-21swapper/019:25:390
13368994821,13cyclictest5538-21sed20:15:512
13368994819,11cyclictest2592-21/usr/sbin/munin23:20:382
13368994819,11cyclictest2592-21/usr/sbin/munin23:20:382
1336799485,4cyclictest23482-21unixbench-2d19:35:521
13366994822,5cyclictest25498-21ssh21:10:490
13366994812,20cyclictest13-21rcuc/019:49:350
1336899477,25cyclictest0-21swapper/219:25:382
13368994719,12cyclictest16399-21taskset19:16:012
13368994716,21cyclictest0-21swapper/219:41:592
13368994716,21cyclictest0-21swapper/219:41:592
13368994710,22cyclictest8050irq/56-dwc_otg_20:23:522
13368994710,22cyclictest8050irq/56-dwc_otg_20:23:522
1336799471,6cyclictest17339-21cut20:50:391
1336799471,10cyclictest0-21swapper/120:34:231
1336699477,21cyclictest0-21swapper/020:10:380
1336699472,33cyclictest0-21swapper/000:13:130
1336699472,22cyclictest0-21swapper/000:20:240
127412473,34sleep00-21swapper/019:05:550
1336999467,12cyclictest0-21swapper/320:35:433
13368994618,19cyclictest17225-21taskset20:50:292
13368994617,16cyclictest7950irq/56-dwc_otg_21:06:142
13368994615,20cyclictest7435-21ntpq23:25:462
13368994615,18cyclictest182592sleep219:21:322
13368994614,22cyclictest0-21swapper/219:47:052
13368994614,11cyclictest0-21swapper/220:26:282
13368994612,18cyclictest29598-21ls19:55:242
1336799463,5cyclictest15224-21aten_rbpower_cu19:15:381
13366994615,15cyclictest727-21runrttasks23:30:570
13368994518,20cyclictest19348-21ssh22:21:222
47072440,2sleep34708-21/usr/sbin/munin20:15:393
1336999444,5cyclictest8050irq/56-dwc_otg_20:30:373
1336999441,10cyclictest0-21swapper/320:10:383
13368994420,11cyclictest21860-21taskset19:31:312
13368994414,12cyclictest13015-21taskset20:36:572
13368994412,18cyclictest0-21swapper/219:39:422
117612440,3sleep10-21swapper/119:05:391
1336699437,23cyclictest0-21swapper/020:20:380
1336699437,23cyclictest0-21swapper/020:20:380
13368994227,12cyclictest16322-21sed20:45:512
13368994217,12cyclictest14108-21/usr/sbin/munin20:40:462
1336699423,25cyclictest13-21rcuc/020:34:290
1336999413,32cyclictest19072-21ssh23:40:353
13368994112,18cyclictest0-21swapper/220:00:392
1336699411,30cyclictest0-21swapper/021:05:290
1336999405,20cyclictest0-21swapper/321:52:553
1336899399,16cyclictest0-21swapper/220:34:402
236172380,2sleep30-21swapper/319:35:543
1336999371,6cyclictest0-21swapper/321:10:353
13369993624,9cyclictest381ktimersoftd/322:05:373
13369993611,5cyclictest0-21swapper/323:32:383
264892351,4sleep326581-21ls19:45:453
1336999354,14cyclictest7950irq/56-dwc_otg_23:20:523
1336999354,14cyclictest7950irq/56-dwc_otg_23:20:523
214722340,3sleep30-21swapper/321:45:413
1336999345,17cyclictest727-21runrttasks21:05:143
1336999344,15cyclictest8050irq/56-dwc_otg_20:00:533
13369993411,5cyclictest37-21rcuc/319:50:513
1336999340,30cyclictest28856-21ssh21:15:373
71812330,2sleep30-21swapper/320:20:503
71812330,2sleep30-21swapper/320:20:503
1336999339,16cyclictest0-21swapper/300:36:143
1336999338,4cyclictest0-21swapper/322:00:393
1336999336,15cyclictest8050irq/56-dwc_otg_20:50:523
1336999336,13cyclictest8050irq/56-dwc_otg_21:40:423
1336999333,8cyclictest750-21snmpd19:55:393
1336999333,28cyclictest0-21swapper/322:45:373
13369993318,4cyclictest727-21runrttasks19:25:243
1336999321,3cyclictest0-21swapper/322:00:113
1336999321,21cyclictest15876-21ls20:45:443
13369993210,10cyclictest0-21swapper/322:17:513
1336999318,10cyclictest0-21swapper/322:20:493
13369993125,4cyclictest15648-21/usr/sbin/munin00:15:443
13369993119,8cyclictest22994-21fschecks_time21:05:423
1336999308,3cyclictest0-21swapper/323:30:223
1336999302,19cyclictest0-21swapper/319:18:303
1336999301,7cyclictest20634-21aten_rbpower_cu19:30:373
1336999291,4cyclictest0-21swapper/319:40:563
1336999291,4cyclictest0-21swapper/319:40:563
1336999291,19cyclictest15311-21latency_hist00:15:243
1336999281,14cyclictest8050irq/56-dwc_otg_19:25:463
1336999281,12cyclictest7850irq/56-dwc_otg20:25:533
13369992620,4cyclictest15212-21latency_hist20:45:243
113972263,13sleep30-21swapper/319:05:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional