You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 14:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri Apr 10, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
166721510,5sleep01054999cyclictest09:16:030
10549991101,72cyclictest0-21swapper/007:05:550
10549999825,60cyclictest7950irq/56-dwc_otg_07:20:410
1054999881,63cyclictest28896-21fschecks_count05:20:580
10549997913,3cyclictest7750irq/56-dwc_otg07:26:060
10549997839,6cyclictest7950irq/56-dwc_otg_08:20:530
10549997839,6cyclictest7950irq/56-dwc_otg_08:20:530
10549997712,50cyclictest307-21unixbench_singl08:36:110
10549997627,46cyclictest101ktimersoftd/005:55:540
1054999761,41cyclictest0-21swapper/005:40:510
10549997422,37cyclictest7850irq/56-dwc_otg_08:56:040
1054999738,12cyclictest0-21swapper/009:40:090
1054999732,54cyclictest0-21swapper/006:58:570
10549997318,6cyclictest7850irq/56-dwc_otg_08:11:080
10549997214,47cyclictest0-21swapper/007:39:590
10549997212,52cyclictest0-21swapper/009:35:340
7312710,2sleep30-21swapper/308:36:483
241072710,2sleep20-21swapper/207:11:002
241072710,2sleep20-21swapper/207:11:002
1054999716,45cyclictest0-21swapper/005:05:550
10549997125,34cyclictest7950irq/56-dwc_otg_09:52:280
1054999711,18cyclictest0-21swapper/009:26:040
1054999702,55cyclictest0-21swapper/007:31:080
10549996931,10cyclictest7950irq/56-dwc_otg_09:55:510
10549996914,4cyclictest7850irq/56-dwc_otg_07:51:050
10549996913,11cyclictest21346-21diskmemload09:04:330
10549996913,11cyclictest21346-21diskmemload09:04:330
10551996825,22cyclictest29964-21rm09:49:302
1054999679,3cyclictest0-21swapper/006:46:090
10549996714,9cyclictest908-21runrttasks04:46:050
10549996714,9cyclictest908-21runrttasks04:46:050
10549996711,45cyclictest0-21swapper/004:51:470
10549996711,35cyclictest0-21swapper/006:31:110
10549996614,6cyclictest0-21swapper/008:10:410
10549996614,4cyclictest27067-21fschecks_count05:15:580
1054999661,19cyclictest5141-21/usr/sbin/munin09:20:510
1054999658,5cyclictest30263-21/usr/sbin/munin06:41:080
1054999656,42cyclictest0-21swapper/004:35:540
1054999652,25cyclictest0-21swapper/009:10:110
10549996520,33cyclictest7950irq/56-dwc_otg_08:30:530
10549996512,5cyclictest7750irq/56-dwc_otg09:45:410
1054999651,19cyclictest0-21swapper/006:14:550
1054999651,19cyclictest0-21swapper/006:14:550
196912640,3sleep00-21swapper/007:43:470
10549996412,8cyclictest25134-21rm08:27:400
1054999641,18cyclictest0-21swapper/005:50:590
1054999641,12cyclictest0-21swapper/007:59:230
1054999632,50cyclictest0-21swapper/007:04:080
1054999632,18cyclictest0-21swapper/009:46:490
1054999632,18cyclictest0-21swapper/004:43:220
1054999631,20cyclictest0-21swapper/008:49:110
1054999631,16cyclictest0-21swapper/004:55:590
10551996227,14cyclictest22503-21/usr/sbin/munin06:31:062
10551996224,14cyclictest25794-21ssh07:50:592
10551996223,22cyclictest1128450rm08:50:152
10551996218,23cyclictest7850irq/56-dwc_otg_09:18:372
1054999622,57cyclictest0-21swapper/006:20:410
1054999622,18cyclictest0-21swapper/007:45:490
1054999622,15cyclictest9-21ksoftirqd/008:51:000
10549996212,10cyclictest0-21swapper/006:36:220
1054999621,17cyclictest5059-21rm08:41:510
1054999621,16cyclictest8861-21rm06:53:080
1054999621,15cyclictest29116-21aten_rbpower_cu09:10:560
1055099613,11cyclictest7750irq/56-dwc_otg08:35:531
1054999612,56cyclictest25577-21systemd-journal05:47:530
1055199609,18cyclictest0-21swapper/207:05:552
10551996026,13cyclictest694-21latency_hist09:15:412
10551996023,12cyclictest15048-21aten_rbpower_po07:00:552
1054999605,13cyclictest0-21swapper/007:21:100
1054999605,13cyclictest0-21swapper/007:21:100
1054999602,20cyclictest20833-21grep06:26:070
10549996017,30cyclictest12448-21latency_hist06:05:420
10549996014,8cyclictest7950irq/56-dwc_otg_07:12:240
10549996014,8cyclictest7950irq/56-dwc_otg_07:12:240
1054999601,18cyclictest0-21swapper/008:16:410
1055199596,32cyclictest0-21swapper/204:30:532
10551995926,12cyclictest5140-21ssh09:57:562
10551995925,18cyclictest4422-21ssh06:47:512
10551995924,19cyclictest15996-21ssh09:32:412
10551995924,11cyclictest23587-21ssh07:48:062
10551995922,13cyclictest8999-21aten_rbpower_en05:55:552
10551995917,12cyclictest31248-21ssh07:56:332
1054999591,50cyclictest26566-21runrttasks05:15:360
1054999591,19cyclictest0-21swapper/004:32:400
1054999591,14cyclictest0-21swapper/008:03:340
10551995825,13cyclictest19067-21ssh08:58:452
10551995825,13cyclictest16316-21/usr/sbin/munin04:45:582
10551995825,13cyclictest16316-21/usr/sbin/munin04:45:582
10551995823,22cyclictest15937-21ssh08:16:352
10551995823,11cyclictest10882-21ssh08:11:042
10551995721,21cyclictest8401-21ssh09:24:302
1054999576,10cyclictest0-21swapper/005:39:480
1054999572,18cyclictest0-21swapper/005:26:400
1054999571,12cyclictest0-21swapper/006:06:470
10551995625,17cyclictest32557-21sed08:36:092
10551995622,10cyclictest14404-21df04:40:552
10551995619,19cyclictest10689-21/usr/sbin/munin09:26:092
10551995617,18cyclictest7950irq/56-dwc_otg_09:35:522
10551995523,18cyclictest25051-21ssh08:27:322
10551995522,11cyclictest29703-21sh08:33:112
10551995521,20cyclictest12164-21taskset06:04:022
10551995520,10cyclictest7850irq/56-dwc_otg_07:20:592
10551995520,10cyclictest7850irq/56-dwc_otg_07:20:592
10551995519,12cyclictest19636-21sed06:25:412
10551995515,13cyclictest6603-21chrt08:44:322
10551995511,19cyclictest7850irq/56-dwc_otg_06:26:072
1054999551,15cyclictest1125-21taskset05:31:420
10549995510,8cyclictest0-21swapper/005:02:120
10551995425,13cyclictest11438-21awk06:56:032
10551995424,19cyclictest27308-21ssh09:08:132
10551995423,17cyclictest21346-21diskmemload06:53:022
10551995423,11cyclictest1223-21ssh06:44:452
10551995421,18cyclictest5104-21rm08:05:022
10551995421,12cyclictest6858-21ssh07:28:482
10551995420,12cyclictest20811-21awk07:45:412
10551995417,23cyclictest14301-21ssh08:53:052
10551995415,25cyclictest0-21swapper/205:26:402
10551995324,17cyclictest22094-21ssh09:01:362
10551995324,17cyclictest22094-21ssh09:01:362
10551995318,10cyclictest7950irq/56-dwc_otg_06:20:402
10551995317,20cyclictest2200-21ssh09:55:112
10551995313,24cyclictest16272-21ssh07:40:162
1055199525,10cyclictest22495-21ssh09:40:562
10551995224,12cyclictest22019-21ssh08:24:422
10551995224,12cyclictest22019-21ssh08:24:422
10551995222,12cyclictest27524-21ssh06:36:322
10551995220,15cyclictest28647-21aten_rbpower_cu05:20:542
1054999521,14cyclictest19308-21runrttasks06:23:050
10551995119,12cyclictest31729-21sh07:20:322
10551995118,13cyclictest10030-21sh07:31:542
10551995116,21cyclictest8105-21sh08:07:502
10551995115,13cyclictest7850irq/56-dwc_otg_05:15:552
10551995018,12cyclictest7750irq/56-dwc_otg05:50:552
10551995012,14cyclictest26342-21sshd05:12:472
10551994818,12cyclictest13943-21chrt04:40:402
84322470,2sleep10-21swapper/109:24:331
153352470,5sleep31055299cyclictest09:31:323
102592472,15sleep00-21swapper/004:30:250
1055299467,24cyclictest0-21swapper/305:00:533
1055099463,11cyclictest7850irq/56-dwc_otg_05:15:521
10551994516,11cyclictest19231-21taskset04:54:042
10551994514,12cyclictest0-21swapper/204:59:462
261392441,39sleep326145-21ssh09:06:123
10551994421,17cyclictest0-21swapper/205:42:252
10551994416,15cyclictest7950irq/56-dwc_otg_05:37:042
10551994413,9cyclictest0-21swapper/205:48:102
305312430,1sleep30-21swapper/309:50:283
1055299437,22cyclictest0-21swapper/304:50:553
1055299431,31cyclictest0-21swapper/309:51:473
10551994315,15cyclictest7950irq/56-dwc_otg_06:06:462
1055099438,24cyclictest0-21swapper/105:25:461
1055099437,23cyclictest0-21swapper/109:40:561
1055099435,5cyclictest19071-21ls09:36:071
10552994218,6cyclictest908-21runrttasks08:56:093
1055199422,6cyclictest0-21swapper/205:05:542
10551994219,10cyclictest15831-21chrt06:14:512
10551994219,10cyclictest15831-21chrt06:14:512
10551994218,9cyclictest7850irq/56-dwc_otg_05:34:412
175112410,2sleep10-21swapper/107:03:181
10552994123,14cyclictest0-21swapper/307:45:533
10552994117,6cyclictest28343-21timerwakeupswit08:31:083
1055299411,17cyclictest7850irq/56-dwc_otg_07:25:523
10551994117,11cyclictest7850irq/56-dwc_otg_05:01:012
1055099417,22cyclictest0-21swapper/108:55:461
10550994114,15cyclictest21346-21diskmemload08:26:081
10552994019,19cyclictest88350ssh09:15:533
1055299401,4cyclictest7950irq/56-dwc_otg_08:05:553
10550994019,6cyclictest23-21ksoftirqd/107:51:021
215522390,3sleep337-21rcuc/309:01:083
215522390,3sleep337-21rcuc/309:01:083
1055299397,25cyclictest0-21swapper/305:30:523
1055299392,30cyclictest7850irq/56-dwc_otg_08:53:223
1055299391,29cyclictest16687-21ls06:16:013
1055099396,21cyclictest24820-21ssh09:05:551
1055099392,26cyclictest0-21swapper/109:20:411
10550993915,10cyclictest23798-21sh06:32:311
1055099391,19cyclictest0-21swapper/108:34:261
129052380,3sleep31055299cyclictest09:29:553
1055299388,7cyclictest7950irq/56-dwc_otg_07:35:533
10552993820,5cyclictest27588-21sh07:52:373
10550993813,6cyclictest16520-21latency_hist07:40:411
10550993810,11cyclictest7950irq/56-dwc_otg_09:10:591
1055299377,24cyclictest0-21swapper/305:05:553
1055299377,18cyclictest7950irq/56-dwc_otg_06:30:543
10552993719,11cyclictest381ktimersoftd/306:36:083
1055299371,25cyclictest18931-21ls07:05:413
1055099376,17cyclictest7950irq/56-dwc_otg_06:51:101
1055099374,24cyclictest7850irq/56-dwc_otg_08:04:161
10550993720,11cyclictest7950irq/56-dwc_otg_08:40:541
288322361,28sleep30-21swapper/307:16:093
1055299363,25cyclictest11617-21ls08:50:413
1055299362,7cyclictest7950irq/56-dwc_otg_07:40:533
10552993616,4cyclictest11890-21dhclient-script04:32:453
1055099369,3cyclictest0-21swapper/107:59:531
1055099368,6cyclictest7850irq/56-dwc_otg_06:21:091
1055099364,8cyclictest16994-21/usr/sbin/munin06:16:061
1055099363,7cyclictest7750irq/56-dwc_otg07:10:511
1055099363,7cyclictest7750irq/56-dwc_otg07:10:511
1055099363,6cyclictest7850irq/56-dwc_otg_08:10:521
1055099361,23cyclictest11724-21ls06:56:071
1055299359,13cyclictest7950irq/56-dwc_otg_09:57:413
1055299355,21cyclictest8963-21ls07:31:033
1055299351,3cyclictest7850irq/56-dwc_otg_09:20:533
1055099359,19cyclictest0-21swapper/109:35:331
1055099357,18cyclictest908-21runrttasks07:43:191
1055099356,6cyclictest7750irq/56-dwc_otg04:40:551
1055099353,5cyclictest0-21swapper/107:30:521
10550993517,11cyclictest23-21ksoftirqd/104:40:411
1055099351,26cyclictest5614-21cut05:45:591
1055299349,4cyclictest15408-21unixbench-2d06:11:093
1055299349,4cyclictest15408-21unixbench-2d06:11:093
1055299345,6cyclictest7950irq/56-dwc_otg_06:25:583
1055299342,4cyclictest0-21swapper/308:29:083
10552993411,10cyclictest0-21swapper/309:14:283
1055099347,4cyclictest15619-21/usr/sbin/munin08:16:081
1055099345,20cyclictest32193-21ssh07:20:551
1055099345,20cyclictest32193-21ssh07:20:551
1055099343,20cyclictest7950irq/56-dwc_otg_05:41:061
1055099343,19cyclictest7950irq/56-dwc_otg_06:45:411
10550993412,4cyclictest2104-21apt-get06:45:521
10550993411,14cyclictest7950irq/56-dwc_otg_08:55:401
92832335,16sleep19324-21sed04:26:011
1055299335,19cyclictest908-21runrttasks07:56:173
1055299335,18cyclictest15453-21kworker/3:104:55:533
1055299334,6cyclictest7850irq/56-dwc_otg_06:40:593
1055299332,6cyclictest7850irq/56-dwc_otg_07:21:113
1055299332,6cyclictest7850irq/56-dwc_otg_07:21:113
10552993316,6cyclictest7440-21ssh06:51:103
1055299331,21cyclictest8801-21ls05:55:413
10552993311,10cyclictest0-21swapper/308:11:413
10552993310,13cyclictest27503-21apt-config07:15:413
10552993310,13cyclictest27503-21apt-config07:15:413
1055099334,14cyclictest7950irq/56-dwc_otg_05:05:401
1055099333,5cyclictest7750irq/56-dwc_otg06:35:591
1055099333,15cyclictest28209-21fschecks_count07:16:001
10550993323,7cyclictest19933-21fschecks_time04:55:581
1055099331,4cyclictest7750irq/56-dwc_otg09:55:531
10550993312,5cyclictest23-21ksoftirqd/107:50:421
150232320,1sleep30-21swapper/304:41:063
1055299329,12cyclictest0-21swapper/309:35:553
1055299326,6cyclictest1676-21tr08:00:553
1055299323,4cyclictest7950irq/56-dwc_otg_05:55:543
10552993220,8cyclictest0-21swapper/309:41:063
10552993215,10cyclictest16298-21grep04:45:573
10552993215,10cyclictest16298-21grep04:45:573
10552993214,5cyclictest37-21rcuc/307:06:033
1055299321,24cyclictest14686-21ls07:00:413
1055099328,4cyclictest7950irq/56-dwc_otg_05:10:521
1055099328,10cyclictest0-21swapper/107:10:411
1055099326,11cyclictest0-21swapper/105:05:561
1055099325,7cyclictest7950irq/56-dwc_otg_05:30:531
1055099323,10cyclictest7850irq/56-dwc_otg_09:01:091
1055099323,10cyclictest7850irq/56-dwc_otg_09:01:091
1055099322,8cyclictest5837-21apt-get08:05:571
1055099322,15cyclictest7950irq/56-dwc_otg_05:36:031
10550993214,6cyclictest7735-21aten_rbpower_vo08:45:561
1055099321,21cyclictest15487-21grep06:11:091
1055099321,21cyclictest15487-21grep06:11:091
10550993211,5cyclictest7850irq/56-dwc_otg_09:45:561
119922311,4sleep10-21swapper/104:33:561
1055299317,15cyclictest0-21swapper/305:20:543
1055299312,5cyclictest14641-21cut08:15:573
1055299312,13cyclictest7950irq/56-dwc_otg_08:40:553
1055299311,16cyclictest19425-21ntp_states08:21:033
1055299311,16cyclictest19425-21ntp_states08:21:033
1055099317,4cyclictest7950irq/56-dwc_otg_04:51:081
1055099315,14cyclictest7950irq/56-dwc_otg_05:55:411
1055099314,12cyclictest7950irq/56-dwc_otg_05:56:441
1055099313,5cyclictest0-21swapper/109:29:211
1055099313,4cyclictest0-21swapper/106:00:521
1055099311,7cyclictest7750irq/56-dwc_otg09:51:011
100372313,21sleep30-21swapper/304:27:323
1055099305,5cyclictest7950irq/56-dwc_otg_06:26:021
1055099305,15cyclictest19715-21timerandwakeup08:21:071
1055099305,15cyclictest19715-21timerandwakeup08:21:071
1055099303,6cyclictest4267-21ssh07:25:571
1055099301,17cyclictest12719-21ls06:05:561
1055299299,4cyclictest0-21swapper/306:49:533
1055299297,4cyclictest7950irq/56-dwc_otg_06:00:583
1055299292,7cyclictest0-21swapper/305:16:263
10552992912,4cyclictest1901-21cpuspeed05:35:563
1055299282,16cyclictest0-21swapper/305:11:273
1055299281,17cyclictest0-21swapper/304:36:223
1055299281,12cyclictest7950irq/56-dwc_otg_06:05:553
1055099287,7cyclictest0-21swapper/105:20:571
87412275,13sleep28796-21/usr/sbin/munin04:25:532
1055299272,8cyclictest0-21swapper/306:25:413
1055099274,5cyclictest7850irq/56-dwc_otg_04:45:581
1055099274,5cyclictest7850irq/56-dwc_otg_04:45:581
1055299267,16cyclictest0-21swapper/305:25:463
1055299261,5cyclictest6428-21sed05:46:103
1055299255,6cyclictest7950irq/56-dwc_otg_05:45:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional