You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 04:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu Jun 04, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
184358922530,13sleep20-21swapper/219:05:162
188959521260,3sleep00-21swapper/021:55:130
186267421220,3sleep33056-21PK-Backend20:20:123
186142321200,4sleep11861424-21latency_hist20:15:011
194543121090,3sleep33056-21PK-Backend00:30:223
184318727649,16sleep10-21swapper/119:05:111
184450826545,14sleep00-21swapper/019:08:040
184431926343,13sleep30-21swapper/319:05:383
18527452510,4sleep10-21swapper/119:40:081
139099340,4rtkit-daemon1389-21rtkit-daemon23:18:372
139099290,4rtkit-daemon1389-21rtkit-daemon23:30:293
139099280,3rtkit-daemon1389-21rtkit-daemon23:21:482
184483899274,3cyclictest141rcu_preempt00:08:403
1844836992618,7cyclictest0-21swapper/222:30:012
19066522250,9sleep342-21ksoftirqd/322:44:593
18701592250,5chrt1870161-21needreboot20:50:141
1844829992510,8cyclictest13-21ksoftirqd/021:07:180
1844829992510,8cyclictest13-21ksoftirqd/019:27:130
1844829992510,4cyclictest13-21ksoftirqd/023:46:080
1844829992510,4cyclictest13-21ksoftirqd/021:49:120
1844829992510,4cyclictest13-21ksoftirqd/021:49:120
184483899240,3cyclictest141rcu_preempt20:43:123
184483699240,22cyclictest0-21swapper/221:52:152
184482999249,4cyclictest13-21ksoftirqd/023:26:560
184482999249,4cyclictest13-21ksoftirqd/022:59:340
184482999249,3cyclictest13-21ksoftirqd/023:50:180
139099240,4rtkit-daemon1389-21rtkit-daemon22:55:383
1844838992310,2cyclictest141rcu_preempt23:15:213
184483699230,22cyclictest1389-21rtkit-daemon23:09:452
184483699230,21cyclictest1909279-21hddtemp_smartct22:50:152
184482999239,8cyclictest13-21ksoftirqd/020:16:360
184482999239,4cyclictest13-21ksoftirqd/022:18:150
184482999238,5cyclictest13-21ksoftirqd/023:43:450
184482999238,4cyclictest13-21ksoftirqd/023:59:570
184482999238,4cyclictest13-21ksoftirqd/021:13:420
184482999238,4cyclictest13-21ksoftirqd/021:03:130
1844829992321,1cyclictest0-21swapper/021:32:430
18931322220,2sleep00-21swapper/022:05:140
184483899223,9cyclictest42-21ksoftirqd/321:38:393
1844838992217,3cyclictest42-21ksoftirqd/322:30:013
1844836992217,4cyclictest1932264-21expr23:55:122
1844836992214,6cyclictest1934801-21sensors_temp00:00:182
1844833992220,1cyclictest3056-21PK-Backend23:17:011
1844833992220,1cyclictest0-21swapper/119:25:021
1844833992220,1cyclictest0-21swapper/100:37:241
1844833992216,4cyclictest26-21ksoftirqd/121:20:081
184482999228,4cyclictest13-21ksoftirqd/019:23:560
184482999228,3cyclictest13-21ksoftirqd/021:52:130
184482999227,4cyclictest13-21ksoftirqd/021:15:440
184482999227,4cyclictest13-21ksoftirqd/000:23:170
184482999226,4cyclictest13-21ksoftirqd/019:37:000
184482999225,3cyclictest13-21ksoftirqd/023:20:030
1844829992220,1cyclictest0-21swapper/022:38:180
1844829992220,1cyclictest0-21swapper/022:21:230
1844829992220,1cyclictest0-21swapper/019:59:210
1844829992216,4cyclictest436-20systemd-journal22:30:010
184482999220,3cyclictest1892485-21munin-run22:05:000
184483899215,13cyclictest42-21ksoftirqd/323:35:043
184483899213,2cyclictest141rcu_preempt22:05:173
1844838992116,4cyclictest42-21ksoftirqd/322:50:003
1844838992116,4cyclictest42-21ksoftirqd/320:00:003
1844838992115,5cyclictest42-21ksoftirqd/320:55:003
1844838992115,4cyclictest1933394-21cron23:59:593
1844838992113,6cyclictest1874511-21apt21:10:083
184483899210,8cyclictest1854123-21munin-run19:45:003
184483899210,4cyclictest42-21ksoftirqd/323:25:163
184483899210,20cyclictest0-21swapper/323:53:283
184483899210,20cyclictest0-21swapper/319:50:093
184483899210,20cyclictest0-21swapper/319:50:093
184483899210,19cyclictest3056-21PK-Backend20:45:523
1844836992119,1cyclictest0-21swapper/221:55:072
1844836992119,1cyclictest0-21swapper/221:44:592
1844836992117,2cyclictest1897778-21gpgv22:20:012
1844836992114,5cyclictest1851899-21date19:35:012
1844836992111,5cyclictest1940064-21cat00:15:222
184483699210,20cyclictest0-21swapper/221:13:232
1844833992119,1cyclictest3056-21PK-Backend23:58:241
1844833992119,1cyclictest3056-21PK-Backend22:28:541
1844833992119,1cyclictest3056-21PK-Backend00:22:101
1844833992119,1cyclictest0-21swapper/122:10:291
1844833992119,1cyclictest0-21swapper/121:59:331
1844833992119,1cyclictest0-21swapper/121:45:281
1844833992119,1cyclictest0-21swapper/121:45:281
1844833992119,1cyclictest0-21swapper/120:36:271
1844833992119,1cyclictest0-21swapper/100:32:311
1844833992119,1cyclictest0-21swapper/100:19:201
1844833992119,1cyclictest0-21swapper/100:11:491
1844833992113,6cyclictest1862513-21fw_packets20:20:111
184483399210,20cyclictest0-21swapper/120:17:211
184483399210,20cyclictest0-21swapper/119:12:041
184483399210,19cyclictest3056-21PK-Backend19:47:471
184483399210,19cyclictest0-21swapper/122:05:021
184482999218,6cyclictest13-21ksoftirqd/020:09:220
184482999218,5cyclictest13-21ksoftirqd/020:47:240
184482999218,4cyclictest13-21ksoftirqd/022:12:180
184482999218,4cyclictest13-21ksoftirqd/019:31:180
184482999218,4cyclictest13-21ksoftirqd/000:16:080
184482999218,3cyclictest13-21ksoftirqd/020:40:420
184482999217,4cyclictest13-21ksoftirqd/000:14:360
184482999216,4cyclictest13-21ksoftirqd/023:39:560
184482999216,4cyclictest13-21ksoftirqd/023:16:060
184482999216,4cyclictest13-21ksoftirqd/023:14:400
184482999216,4cyclictest13-21ksoftirqd/022:47:070
184482999216,4cyclictest13-21ksoftirqd/000:25:590
184482999216,4cyclictest13-21ksoftirqd/000:05:420
184482999216,3cyclictest13-21ksoftirqd/021:20:100
184482999214,14cyclictest1947546-21cron00:40:000
1844829992119,1cyclictest0-21swapper/022:40:210
1844829992119,1cyclictest0-21swapper/021:41:340
1844829992119,1cyclictest0-21swapper/020:52:140
1844829992119,1cyclictest0-21swapper/020:13:340
1844829992119,1cyclictest0-21swapper/020:02:460
1844829992119,1cyclictest0-21swapper/019:46:570
1844829992119,1cyclictest0-21swapper/019:11:210
1844829992118,2cyclictest0-21swapper/020:30:030
1844829992118,2cyclictest0-21swapper/000:30:130
1844829992114,5cyclictest13-21ksoftirqd/021:35:230
1844829992113,6cyclictest1934644-21awk00:00:160
1844829992110,2cyclictest141rcu_preempt19:55:000
1844829992110,2cyclictest141rcu_preempt19:55:000
184483899209,9cyclictest1850045-21unin-run19:30:013
184483899209,4cyclictest42-21ksoftirqd/321:52:113
184483899209,4cyclictest42-21ksoftirqd/320:59:293
184483899208,3cyclictest42-21ksoftirqd/323:23:083
184483899204,3cyclictest141rcu_preempt20:10:013
1844838992018,1cyclictest0-21swapper/322:16:093
1844838992017,2cyclictest3056-21PK-Backend22:52:463
1844838992015,4cyclictest42-21ksoftirqd/319:40:013
1844838992013,5cyclictest1938756-21sshd00:13:403
1844838992013,5cyclictest1913535-21sshd23:01:393
1844838992011,4cyclictest42-21ksoftirqd/300:16:463
184483899200,19cyclictest1389-21rtkit-daemon22:32:393
184483899200,19cyclictest0-21swapper/323:42:073
184483899200,19cyclictest0-21swapper/323:08:143
184483899200,19cyclictest0-21swapper/319:31:363
184483699209,9cyclictest1857145-21unin-run20:00:002
1844836992018,1cyclictest0-21swapper/223:45:452
1844836992018,1cyclictest0-21swapper/222:35:082
1844836992018,1cyclictest0-21swapper/222:08:522
1844836992018,1cyclictest0-21swapper/221:09:102
1844836992018,1cyclictest0-21swapper/220:50:442
1844836992018,1cyclictest0-21swapper/220:39:572
1844836992018,1cyclictest0-21swapper/220:07:322
1844836992018,1cyclictest0-21swapper/219:25:342
1844836992015,3cyclictest1926485-21idleruntime-cro23:40:002
1844836992013,6cyclictest1936397-21ntp_kernel_pll_00:05:192
184483699200,19cyclictest3056-21PK-Backend00:10:332
184483699200,19cyclictest0-21swapper/223:34:022
184483699200,19cyclictest0-21swapper/223:04:492
184483699200,19cyclictest0-21swapper/221:28:502
184483699200,19cyclictest0-21swapper/220:46:322
184483699200,19cyclictest0-21swapper/220:18:472
184483699200,19cyclictest0-21swapper/220:04:552
184483699200,19cyclictest0-21swapper/219:48:532
184483699200,19cyclictest0-21swapper/219:11:222
1844833992018,1cyclictest3056-21PK-Backend23:13:571
1844833992018,1cyclictest3056-21PK-Backend22:50:041
1844833992018,1cyclictest3056-21PK-Backend22:15:571
1844833992018,1cyclictest3056-21PK-Backend21:15:201
1844833992018,1cyclictest3056-21PK-Backend00:02:161
1844833992018,1cyclictest0-21swapper/120:40:261
1844833992018,1cyclictest0-21swapper/120:30:241
1844833992018,1cyclictest0-21swapper/120:04:431
1844833992018,1cyclictest0-21swapper/119:30:011
1844833992012,6cyclictest1891760-21ntpq22:00:191
184483399200,19cyclictest3056-21PK-Backend23:53:311
184483399200,19cyclictest3056-21PK-Backend23:27:011
184483399200,19cyclictest3056-21PK-Backend19:51:251
184483399200,19cyclictest3056-21PK-Backend19:51:251
184483399200,19cyclictest0-21swapper/121:41:021
184483399200,19cyclictest0-21swapper/121:39:131
184483399200,19cyclictest0-21swapper/120:45:121
184483399200,19cyclictest0-21swapper/120:25:581
184483399200,19cyclictest0-21swapper/100:09:461
184483399200,18cyclictest3056-21PK-Backend22:32:461
184482999208,3cyclictest13-21ksoftirqd/020:24:300
184482999207,3cyclictest13-21ksoftirqd/023:30:410
1844829992010,4cyclictest13-21ksoftirqd/021:29:590
184482999200,6cyclictest13-21ksoftirqd/023:05:430
184482999200,19cyclictest0-21swapper/022:31:230
184482999200,19cyclictest0-21swapper/020:38:280
139099200,3rtkit-daemon1389-21rtkit-daemon22:12:342
139099200,3rtkit-daemon1389-21rtkit-daemon21:03:452
184483899197,2cyclictest141rcu_preempt21:01:023
1844838991917,1cyclictest3056-21PK-Backend20:25:033
1844838991917,1cyclictest0-21swapper/322:36:103
1844838991917,1cyclictest0-21swapper/322:20:223
1844838991915,3cyclictest42-21ksoftirqd/300:00:013
1844838991914,4cyclictest42-21ksoftirqd/321:50:003
1844838991914,4cyclictest42-21ksoftirqd/321:49:593
1844838991913,4cyclictest1940879-21basename00:20:113
1844838991912,5cyclictest1942828-21cstates00:25:133
184483899190,1cyclictest3056-21PK-Backend19:11:083
184483899190,1cyclictest3056-21PK-Backend00:37:303
184483899190,1cyclictest0-21swapper/323:47:003
184483899190,1cyclictest0-21swapper/319:46:293
184483899190,1cyclictest0-21swapper/319:21:363
184483899190,18cyclictest1916524-21/usr/sbin/munin23:10:153
1844836991917,1cyclictest0-21swapper/223:28:412
1844836991917,1cyclictest0-21swapper/223:12:262
1844836991917,1cyclictest0-21swapper/222:49:142
1844836991917,1cyclictest0-21swapper/222:31:032
1844836991917,1cyclictest0-21swapper/222:21:492
1844836991917,1cyclictest0-21swapper/222:01:362
1844836991917,1cyclictest0-21swapper/221:38:442
1844836991917,1cyclictest0-21swapper/221:15:122
1844836991917,1cyclictest0-21swapper/219:23:442
1844836991916,2cyclictest3056-21PK-Backend21:22:002
1844836991913,5cyclictest1853387-21ntp_states19:40:152
1844836991912,6cyclictest1846969-21cpuspeed_turbos19:15:022
1844836991912,5cyclictest34-21ksoftirqd/223:50:182
1844836991912,5cyclictest34-21ksoftirqd/200:20:232
184483699190,1cyclictest3056-21PK-Backend20:30:302
184483699190,1cyclictest0-21swapper/221:30:402
184483699190,1cyclictest0-21swapper/220:26:522
184483699190,1cyclictest0-21swapper/220:20:372
184483699190,1cyclictest0-21swapper/219:53:462
184483699190,1cyclictest0-21swapper/219:53:462
184483699190,1cyclictest0-21swapper/219:36:292
184483699190,1cyclictest0-21swapper/200:30:332
184483699190,1cyclictest0-21swapper/200:27:282
184483699190,18cyclictest0-21swapper/221:46:492
184483699190,18cyclictest0-21swapper/221:46:482
184483699190,18cyclictest0-21swapper/220:56:332
184483399196,11cyclictest1871910-21unin-run21:00:001
1844833991917,1cyclictest3056-21PK-Backend23:47:501
1844833991917,1cyclictest3056-21PK-Backend23:21:061
1844833991917,1cyclictest3056-21PK-Backend23:05:581
1844833991917,1cyclictest3056-21PK-Backend22:20:231
1844833991917,1cyclictest3056-21PK-Backend19:37:051
1844833991917,1cyclictest0-21swapper/121:29:331
184483399190,1cyclictest3056-21PK-Backend23:42:561
184483399190,1cyclictest3056-21PK-Backend23:01:521
184483399190,1cyclictest3056-21PK-Backend22:38:281
184483399190,1cyclictest0-21swapper/122:56:461
184483399190,1cyclictest0-21swapper/122:42:201
184483399190,1cyclictest0-21swapper/121:32:091
184483399190,1cyclictest0-21swapper/119:57:531
184482999198,2cyclictest141rcu_preempt19:17:080
184482999197,3cyclictest13-21ksoftirqd/020:28:150
1844829991917,1cyclictest0-21swapper/022:52:120
1844829991917,1cyclictest0-21swapper/019:42:160
1844829991915,3cyclictest13-21ksoftirqd/020:59:300
184482999190,10cyclictest13-21ksoftirqd/023:01:420
184483899184,3cyclictest141rcu_preempt19:18:373
184483899182,14cyclictest1874447-21latency_hist21:10:013
1844838991816,1cyclictest3056-21PK-Backend20:12:333
1844838991815,2cyclictest3056-21PK-Backend21:25:543
184483899181,3cyclictest141rcu_preempt22:00:523
184483899181,3cyclictest141rcu_preempt21:43:403
184483899181,3cyclictest141rcu_preempt20:36:193
184483899181,3cyclictest141rcu_preempt20:17:393
1844838991813,3cyclictest1858114-21ntp_states20:00:143
184483899181,2cyclictest3056-21PK-Backend21:17:243
1844838991812,5cyclictest42-21ksoftirqd/321:55:113
1844836991816,1cyclictest0-21swapper/222:58:262
1844836991816,1cyclictest0-21swapper/200:35:412
1844833991816,1cyclictest3056-21PK-Backend23:32:081
1844833991816,1cyclictest3056-21PK-Backend21:09:321
184483399180,17cyclictest3056-21PK-Backend21:53:191
184483899171,3cyclictest141rcu_preempt21:30:333
184483899171,2cyclictest141rcu_preempt22:10:093
184483899170,2cyclictest141rcu_preempt20:34:593
1844836991710,5cyclictest883-21dbus-daemon20:15:002
1844836991710,5cyclictest1927458-21ntpq23:40:182
184483399179,6cyclictest1847051-21df19:15:031
184483399175,2cyclictest141rcu_preempt23:38:101
1844833991715,1cyclictest3056-21PK-Backend22:49:021
1844833991715,1cyclictest0-21swapper/120:09:431
184483899161,2cyclictest141rcu_preempt21:21:343
1844836991614,1cyclictest0-21swapper/222:40:152
184483399159,5cyclictest1872477-21cat21:00:081
184483399159,5cyclictest1848108-21ls19:20:091
1844833991410,3cyclictest3056-21PK-Backend00:27:231
184483699130,11cyclictest1867612-21sed20:40:122
184483399138,3cyclictest1875034-21latency21:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional