You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 21:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Aug 02, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212708721400,3sleep33056-21PK-Backend11:55:003
212087121250,2sleep10-21swapper/111:35:171
208024621220,3sleep30-21swapper/309:40:123
208024621220,3sleep30-21swapper/309:40:113
205201921120,4sleep33056-21PK-Backend08:20:193
202500227947,21sleep10-21swapper/107:05:271
21259992750,3sleep20-21swapper/211:50:192
202461127456,9sleep00-21swapper/007:05:170
202545326840,22sleep20-21swapper/207:06:522
202545926241,14sleep30-21swapper/307:06:573
21265022460,2chrt0-21swapper/011:50:240
21170112400,2sleep33056-21PK-Backend11:25:133
2025851993430,3cyclictest13-21ksoftirqd/008:26:450
139099330,4rtkit-daemon1317-21systemd12:27:332
2025857993217,5cyclictest436-20systemd-journal12:00:002
139099320,2rtkit-daemon0-21swapper/208:57:432
139099310,3rtkit-daemon1317-21systemd12:30:552
139099300,4rtkit-daemon1317-21systemd11:34:092
21167022290,4sleep22025846-21cyclictest11:25:122
20461062290,3sleep00-21swapper/008:05:000
2025859992617,5cyclictest2032056-21apt07:25:113
2025856992610,3cyclictest26-21ksoftirqd/107:22:381
139099260,4rtkit-daemon1389-21rtkit-daemon12:39:132
139099260,4rtkit-daemon1389-21rtkit-daemon12:35:153
202585999252,6cyclictest2110582-21python311:05:213
2025859992513,9cyclictest883-21dbus-daemon08:41:473
202585699258,3cyclictest141rcu_preempt10:45:051
202585699258,3cyclictest141rcu_preempt07:35:071
202585699251,22cyclictest2071914-21python309:15:181
2025851992520,3cyclictest2075645-21python309:25:200
2025857992415,7cyclictest883-21dbus-daemon08:41:462
2025857992413,9cyclictest2108098-21/usr/sbin/munin11:00:132
202585699249,3cyclictest26-21ksoftirqd/110:41:391
202585699249,3cyclictest26-21ksoftirqd/108:53:521
202585699249,3cyclictest26-21ksoftirqd/108:53:521
202585699248,2cyclictest141rcu_preempt11:30:281
202585699248,2cyclictest141rcu_preempt07:55:271
202585699243,20cyclictest886-21NetworkManager07:14:061
2025851992416,6cyclictest2055781-21python308:30:220
2025851992416,6cyclictest2040508-21apt-get07:50:010
2025859992320,2cyclictest3056-21PK-Backend10:33:383
202585999231,8cyclictest2060362-21/usr/sbin/munin08:45:133
2025857992314,7cyclictest2094491-21df_inode10:20:142
202585699235,4cyclictest26-21ksoftirqd/108:07:071
202585699230,11cyclictest26-21ksoftirqd/111:00:211
202585199236,3cyclictest13-21ksoftirqd/009:14:160
139099230,4rtkit-daemon1389-21rtkit-daemon11:39:192
139099230,4rtkit-daemon1389-21rtkit-daemon10:39:392
139099230,4rtkit-daemon1389-21rtkit-daemon09:11:362
20363902220,4sleep03056-21PK-Backend07:35:210
2025859992220,1cyclictest3056-21PK-Backend09:38:223
2025859992211,8cyclictest2123573-21cron11:45:013
202585999220,20cyclictest3056-21PK-Backend09:19:433
2025857992220,1cyclictest0-21swapper/209:31:352
2025857992220,1cyclictest0-21swapper/208:36:132
202585699227,3cyclictest26-21ksoftirqd/112:26:311
202585699224,5cyclictest26-21ksoftirqd/112:15:131
2025856992220,1cyclictest0-21swapper/111:22:471
2025856992220,1cyclictest0-21swapper/107:47:311
2025856992213,5cyclictest1095-21runrttasks10:00:231
2025856992210,2cyclictest141rcu_preempt09:26:011
2025856992210,2cyclictest141rcu_preempt08:36:021
202585699220,6cyclictest3056-21PK-Backend12:35:161
202585699220,21cyclictest2095643-21date10:25:011
202585699220,21cyclictest0-21swapper/110:05:551
202585199229,5cyclictest13-21ksoftirqd/012:15:130
2025851992215,5cyclictest2111170-21cron11:10:010
2025851992210,4cyclictest13-21ksoftirqd/012:40:010
139099220,4rtkit-daemon1389-21rtkit-daemon10:14:312
139099220,4rtkit-daemon1389-21rtkit-daemon09:56:222
139099220,4rtkit-daemon1389-21rtkit-daemon09:47:522
139099220,4rtkit-daemon1389-21rtkit-daemon09:42:312
139099220,4rtkit-daemon1389-21rtkit-daemon09:42:312
139099220,4rtkit-daemon1389-21rtkit-daemon09:18:162
139099220,4rtkit-daemon1389-21rtkit-daemon08:29:102
139099220,4rtkit-daemon1389-21rtkit-daemon07:44:492
139099220,3rtkit-daemon1389-21rtkit-daemon08:31:022
2025859992119,1cyclictest3056-21PK-Backend12:17:273
2025859992119,1cyclictest3056-21PK-Backend12:00:443
2025859992119,1cyclictest3056-21PK-Backend09:21:063
2025859992119,1cyclictest3056-21PK-Backend09:13:553
2025859992119,1cyclictest3056-21PK-Backend08:28:193
2025859992119,1cyclictest3056-21PK-Backend07:38:033
2025859992119,1cyclictest3056-21PK-Backend07:12:283
2025859992119,1cyclictest0-21swapper/309:03:343
2025859992119,1cyclictest0-21swapper/307:19:453
202585999210,20cyclictest3056-21PK-Backend10:50:553
202585999210,19cyclictest3056-21PK-Backend10:57:313
202585999210,19cyclictest3056-21PK-Backend09:48:293
2025857992119,1cyclictest0-21swapper/212:23:252
2025857992119,1cyclictest0-21swapper/212:11:082
2025857992119,1cyclictest0-21swapper/211:42:292
2025857992119,1cyclictest0-21swapper/211:24:582
2025857992119,1cyclictest0-21swapper/211:18:572
2025857992119,1cyclictest0-21swapper/210:47:402
2025857992119,1cyclictest0-21swapper/210:44:562
2025857992119,1cyclictest0-21swapper/209:27:312
2025857992119,1cyclictest0-21swapper/207:49:492
2025857992119,1cyclictest0-21swapper/207:26:382
2025857992113,6cyclictest2048830-21python308:10:202
202585799210,20cyclictest0-21swapper/211:48:302
202585799210,20cyclictest0-21swapper/210:28:132
202585799210,20cyclictest0-21swapper/210:16:442
202585799210,20cyclictest0-21swapper/209:24:462
202585799210,20cyclictest0-21swapper/208:51:132
202585799210,20cyclictest0-21swapper/208:51:132
202585799210,1cyclictest0-21swapper/210:02:382
202585699219,2cyclictest141rcu_preempt08:40:341
2025856992119,1cyclictest3056-21PK-Backend08:17:121
2025856992119,1cyclictest0-21swapper/112:30:271
2025856992119,1cyclictest0-21swapper/112:10:121
2025856992119,1cyclictest0-21swapper/112:03:231
2025856992119,1cyclictest0-21swapper/111:53:151
2025856992119,1cyclictest0-21swapper/110:13:321
2025856992119,1cyclictest0-21swapper/109:35:541
2025856992119,1cyclictest0-21swapper/109:33:101
2025856992119,1cyclictest0-21swapper/108:25:301
2025856992110,2cyclictest141rcu_preempt09:06:051
202585699210,20cyclictest3056-21PK-Backend07:31:561
202585699210,20cyclictest0-21swapper/111:42:341
202585699210,20cyclictest0-21swapper/109:45:261
202585699210,19cyclictest3056-21PK-Backend11:07:051
202585199218,4cyclictest13-21ksoftirqd/009:08:450
2025851992113,6cyclictest2079985-21perf09:40:000
202585199210,20cyclictest0-21swapper/011:21:540
202585199210,20cyclictest0-21swapper/007:19:080
139099210,4rtkit-daemon1389-21rtkit-daemon11:35:223
139099210,4rtkit-daemon1389-21rtkit-daemon10:34:552
139099210,3rtkit-daemon1389-21rtkit-daemon12:08:072
2025859992018,1cyclictest3056-21PK-Backend11:24:143
2025859992018,1cyclictest3056-21PK-Backend10:49:333
2025859992018,1cyclictest3056-21PK-Backend10:20:403
2025859992018,1cyclictest0-21swapper/310:41:373
2025859992018,1cyclictest0-21swapper/307:52:303
2025859992018,1cyclictest0-21swapper/307:32:023
2025859992017,2cyclictest3056-21PK-Backend12:23:293
2025859992012,6cyclictest2062270-21grep08:50:133
2025859992012,6cyclictest2062270-21grep08:50:133
2025859992011,7cyclictest2067593-21apt09:05:133
202585999200,1cyclictest0-21swapper/308:16:443
202585999200,19cyclictest3056-21PK-Backend11:58:013
202585999200,19cyclictest3056-21PK-Backend09:30:583
202585999200,19cyclictest3056-21PK-Backend08:12:583
202585999200,19cyclictest0-21swapper/311:03:193
202585999200,19cyclictest0-21swapper/309:28:153
202585999200,19cyclictest0-21swapper/308:09:133
202585999200,19cyclictest0-21swapper/307:20:203
202585999200,18cyclictest3056-21PK-Backend12:14:453
2025857992018,1cyclictest0-21swapper/212:16:352
2025857992018,1cyclictest0-21swapper/211:14:162
2025857992018,1cyclictest0-21swapper/210:55:512
2025857992018,1cyclictest0-21swapper/210:50:242
202585799200,1cyclictest0-21swapper/211:09:372
202585799200,19cyclictest0-21swapper/208:07:202
202585799200,19cyclictest0-21swapper/207:51:592
202585799200,19cyclictest0-21swapper/207:17:312
202585699209,2cyclictest141rcu_preempt07:53:061
2025856992018,1cyclictest0-21swapper/112:20:191
2025856992018,1cyclictest0-21swapper/108:21:381
2025856992017,2cyclictest3056-21PK-Backend08:02:261
2025856992017,2cyclictest0-21swapper/111:55:591
2025856992013,5cyclictest2048128-21grep08:10:111
202585699200,6cyclictest3056-21PK-Backend07:15:191
202585699200,1cyclictest3056-21PK-Backend07:28:381
202585699200,19cyclictest0-21swapper/112:06:061
202585699200,19cyclictest0-21swapper/109:20:321
202585699200,19cyclictest0-21swapper/107:40:421
202585699200,10cyclictest26-21ksoftirqd/110:50:191
202585199208,4cyclictest13-21ksoftirqd/011:35:010
202585199208,4cyclictest13-21ksoftirqd/011:30:080
202585199206,4cyclictest13-21ksoftirqd/009:20:020
202585199204,14cyclictest2049425-21sh08:15:010
2025851992018,1cyclictest0-21swapper/012:13:110
2025851992013,5cyclictest2132426-21apt-key12:10:010
2025851992012,6cyclictest2096606-21awk10:25:200
2025851992012,6cyclictest2052513-21users08:20:240
2025851992011,2cyclictest141rcu_preempt07:11:300
202585199200,1cyclictest0-21swapper/009:18:460
202585199200,19cyclictest0-21swapper/011:18:360
202585199200,19cyclictest0-21swapper/009:48:050
202585199200,19cyclictest0-21swapper/009:40:420
202585199200,19cyclictest0-21swapper/009:40:410
202585199200,19cyclictest0-21swapper/008:52:420
202585199200,19cyclictest0-21swapper/008:52:420
202585199200,19cyclictest0-21swapper/008:07:130
202585199200,19cyclictest0-21swapper/007:31:310
139099200,3rtkit-daemon1389-21rtkit-daemon08:21:392
139099200,17rtkit-daemon1317-21systemd10:06:462
21407042190,3sleep03056-21PK-Backend12:30:220
2025859991917,1cyclictest3056-21PK-Backend11:46:453
2025859991917,1cyclictest3056-21PK-Backend11:11:503
2025859991917,1cyclictest0-21swapper/312:30:183
2025859991917,1cyclictest0-21swapper/307:58:593
2025859991916,2cyclictest3056-21PK-Backend07:42:093
202585999190,1cyclictest3056-21PK-Backend12:05:243
202585999190,1cyclictest3056-21PK-Backend11:15:423
202585999190,1cyclictest3056-21PK-Backend10:36:563
202585999190,1cyclictest3056-21PK-Backend09:58:363
202585999190,1cyclictest0-21swapper/311:30:503
202585999190,1cyclictest0-21swapper/310:26:283
202585999190,18cyclictest0-21swapper/308:38:473
202585799190,1cyclictest0-21swapper/209:39:472
202585799190,1cyclictest0-21swapper/207:21:162
202585799190,18cyclictest0-21swapper/208:16:252
2025856991917,1cyclictest0-21swapper/110:57:261
2025856991917,1cyclictest0-21swapper/110:33:331
2025856991917,1cyclictest0-21swapper/109:41:211
2025856991917,1cyclictest0-21swapper/109:41:211
2025856991910,2cyclictest141rcu_preempt08:48:441
202585699190,1cyclictest0-21swapper/110:28:541
202585699190,1cyclictest0-21swapper/109:54:121
202585699190,18cyclictest0-21swapper/108:31:181
202585199197,3cyclictest13-21ksoftirqd/011:47:340
2025851991917,1cyclictest3056-21PK-Backend07:22:190
2025851991917,1cyclictest0-21swapper/011:04:160
2025851991917,1cyclictest0-21swapper/010:30:570
2025851991917,1cyclictest0-21swapper/009:53:320
2025851991917,1cyclictest0-21swapper/009:30:340
2025851991917,1cyclictest0-21swapper/008:40:530
2025851991915,3cyclictest2090385-21perf10:10:000
2025851991914,3cyclictest2130016-21timerandwakeup12:00:240
2025851991912,5cyclictest2094598-21awk10:20:170
2025851991912,5cyclictest2086379-21cat09:55:220
202585199190,1cyclictest0-21swapper/012:21:560
2025859991816,1cyclictest3056-21PK-Backend09:52:353
2025859991816,1cyclictest3056-21PK-Backend08:31:373
2025859991816,1cyclictest0-21swapper/312:26:133
2025859991816,1cyclictest0-21swapper/308:55:573
2025859991811,4cyclictest2040520-21turbostat07:50:003
2025857991813,3cyclictest2035996-21interrupts07:35:162
2025857991811,5cyclictest2042772-21apt07:55:142
202585699188,2cyclictest141rcu_preempt09:01:351
202585699187,2cyclictest141rcu_preempt11:17:321
2025856991816,1cyclictest3056-21PK-Backend08:56:121
2025856991816,1cyclictest0-21swapper/111:29:241
2025856991815,2cyclictest0-21swapper/111:10:231
2025856991810,6cyclictest2099878-21cat10:35:181
2025856991810,6cyclictest2086200-21ntpq09:55:201
202585699180,8cyclictest26-21ksoftirqd/111:45:131
202585199189,7cyclictest2050280-21memory08:15:170
202585199187,3cyclictest13-21ksoftirqd/010:19:260
202585199186,4cyclictest13-21ksoftirqd/010:00:220
202585199185,4cyclictest2065044-21chrt08:56:410
202585199185,4cyclictest2042437-21chrt07:54:220
202585199182,4cyclictest13-21ksoftirqd/008:45:100
2025851991816,1cyclictest0-21swapper/011:57:020
2025851991816,1cyclictest0-21swapper/011:43:370
2025851991816,1cyclictest0-21swapper/011:11:260
2025851991816,1cyclictest0-21swapper/007:40:510
2025851991813,3cyclictest2106446-21cstates10:55:130
2025851991811,5cyclictest2116958-21expr11:25:130
2025851991811,5cyclictest2057289-21cat08:35:180
202585199180,1cyclictest0-21swapper/012:27:580
202585199180,1cyclictest0-21swapper/010:53:140
202585199180,1cyclictest0-21swapper/010:46:390
202585999179,6cyclictest436-20systemd-journal08:01:363
2025859991715,1cyclictest3056-21PK-Backend10:11:423
2025859991711,4cyclictest2093859-21ls10:20:003
2025859991710,5cyclictest2089507-21grep10:05:163
2025857991715,1cyclictest0-21swapper/209:08:512
2025857991715,1cyclictest0-21swapper/208:01:252
2025857991711,4cyclictest2060384-21awk08:45:092
2025857991710,5cyclictest2033955-21grep07:30:142
2025856991715,1cyclictest0-21swapper/109:10:531
202585699170,5cyclictest0-21swapper/110:18:121
2025851991712,3cyclictest2091626-21timerandwakeup10:10:210
2025851991710,5cyclictest2099333-21grep10:35:140
202585199168,6cyclictest2067054-21apt-get09:05:000
202585199163,2cyclictest141rcu_preempt07:58:400
2025851991610,5cyclictest2032323-21expr07:25:120
202585799159,4cyclictest2129182-21grep12:00:142
2025857991511,3cyclictest2026014-21/usr/sbin/munin07:10:182
2025857991511,3cyclictest1095-21runrttasks09:50:262
202585199152,2cyclictest0-21swapper/010:40:020
139099150,12rtkit-daemon3056-21PK-Backend10:00:073
139099150,12rtkit-daemon0-21swapper/209:00:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional