You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 07:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Apr 22, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271235121070,3sleep3411rcuc/320:36:353
276035521060,3sleep20-21swapper/222:55:162
139099840,68rtkit-daemon0-21swapper/319:05:233
268061827049,13sleep10-21swapper/119:10:001
268038326747,14sleep20-21swapper/219:07:012
268027126651,9sleep00-21swapper/019:05:330
27555172540,2sleep10-21swapper/122:40:201
27304592480,2sleep00-21swapper/021:30:130
27710622400,4sleep00-21swapper/023:25:170
27034112350,8sleep0131ksoftirqd/020:10:210
2680779992612,12cyclictest2749308-21idleruntime-cro22:25:010
2680789992511,12cyclictest2777318-21unin-run23:45:013
27634282240,9sleep013-21ksoftirqd/023:05:120
268077999240,22cyclictest2778051-21sed23:45:160
27920842230,2sleep10-21swapper/100:25:131
26940432230,4chrt3056-21PK-Backend19:45:131
268078999230,21cyclictest2682720-21python319:15:023
268078299230,21cyclictest3056-21PK-Backend20:39:391
139099230,15rtkit-daemon0-21swapper/223:39:112
268078999228,12cyclictest2756259-21unin-run22:45:013
2680789992220,1cyclictest3056-21PK-Backend23:30:193
2680789992220,1cyclictest3056-21PK-Backend22:32:403
2680789992220,1cyclictest3056-21PK-Backend21:15:203
2680789992220,1cyclictest0-21swapper/320:11:393
2680789992219,2cyclictest3056-21PK-Backend23:05:253
268078999220,20cyclictest3056-21PK-Backend23:15:123
268078999220,20cyclictest3056-21PK-Backend22:53:503
2680782992220,1cyclictest3056-21PK-Backend21:05:071
2680782992220,1cyclictest3056-21PK-Backend19:25:371
2680782992220,1cyclictest0-21swapper/123:35:201
2680782992214,7cyclictest2780084-21ps23:50:191
2680782992210,9cyclictest2791174-21cron00:25:001
2680779992220,1cyclictest0-21swapper/023:20:560
2680779992220,1cyclictest0-21swapper/021:18:210
139099220,16rtkit-daemon3056-21PK-Backend22:38:513
27058682210,4sleep13056-21PK-Backend20:20:011
2680789992119,1cyclictest3056-21PK-Backend23:27:153
2680789992119,1cyclictest3056-21PK-Backend23:13:093
2680789992119,1cyclictest3056-21PK-Backend22:57:413
2680789992119,1cyclictest3056-21PK-Backend00:17:583
2680789992119,1cyclictest0-21swapper/323:59:333
2680789992119,1cyclictest0-21swapper/322:46:053
2680789992119,1cyclictest0-21swapper/322:05:163
2680789992119,1cyclictest0-21swapper/320:54:323
2680789992119,1cyclictest0-21swapper/320:46:343
2680789992119,1cyclictest0-21swapper/319:51:113
2680789992119,1cyclictest0-21swapper/319:48:553
2680789992119,1cyclictest0-21swapper/319:34:423
2680789992119,1cyclictest0-21swapper/300:02:373
2680789992114,4cyclictest2697679-21diskstats19:55:153
268078999211,12cyclictest27213222awk21:05:013
268078999211,12cyclictest27213222awk21:05:003
268078999210,20cyclictest0-21swapper/319:38:003
268078999210,20cyclictest0-21swapper/319:24:013
2680785992119,1cyclictest0-21swapper/220:19:492
2680785992114,5cyclictest2689795-21sshd19:31:142
2680785992112,7cyclictest2706362-21fw_packets20:20:152
268078599210,20cyclictest0-21swapper/223:41:122
268078599210,20cyclictest0-21swapper/223:14:082
268078599210,19cyclictest0-21swapper/223:05:092
268078299212,17cyclictest2740638-21perf22:00:001
2680782992119,1cyclictest3056-21PK-Backend21:45:431
2680782992119,1cyclictest3056-21PK-Backend20:12:211
2680782992119,1cyclictest3056-21PK-Backend00:37:531
2680782992119,1cyclictest0-21swapper/123:33:171
2680782992119,1cyclictest0-21swapper/123:29:121
2680782992119,1cyclictest0-21swapper/121:11:021
2680782992119,1cyclictest0-21swapper/100:16:371
2680782992119,1cyclictest0-21swapper/100:00:281
2680782992118,2cyclictest3056-21PK-Backend21:25:491
2680782992118,2cyclictest3056-21PK-Backend19:52:201
2680782992111,7cyclictest436-20systemd-journal21:20:011
268078299210,20cyclictest3056-21PK-Backend23:57:241
268078299210,20cyclictest3056-21PK-Backend22:24:501
268078299210,20cyclictest3056-21PK-Backend20:23:231
268078299210,20cyclictest3056-21PK-Backend19:31:591
268078299210,20cyclictest0-21swapper/122:35:591
268078299210,19cyclictest3056-21PK-Backend20:27:151
268078299210,19cyclictest3056-21PK-Backend19:11:391
2680779992119,1cyclictest0-21swapper/023:53:280
2680779992119,1cyclictest0-21swapper/023:15:490
2680779992119,1cyclictest0-21swapper/022:44:260
2680779992119,1cyclictest0-21swapper/022:33:520
2680779992119,1cyclictest0-21swapper/022:01:470
2680779992119,1cyclictest0-21swapper/022:01:470
2680779992119,1cyclictest0-21swapper/021:58:570
2680779992119,1cyclictest0-21swapper/021:37:140
2680779992119,1cyclictest0-21swapper/020:53:540
2680779992119,1cyclictest0-21swapper/020:37:250
2680779992119,1cyclictest0-21swapper/020:33:330
2680779992119,1cyclictest0-21swapper/020:22:450
2680779992119,1cyclictest0-21swapper/020:00:070
2680779992119,1cyclictest0-21swapper/019:40:200
2680779992119,1cyclictest0-21swapper/019:38:030
2680779992119,1cyclictest0-21swapper/019:24:050
2680779992119,1cyclictest0-21swapper/000:39:170
268077999210,20cyclictest0-21swapper/021:41:060
268077999210,20cyclictest0-21swapper/020:55:570
268077999210,20cyclictest0-21swapper/019:58:520
268077999210,20cyclictest0-21swapper/019:17:280
268077999210,20cyclictest0-21swapper/000:33:090
268077999210,20cyclictest0-21swapper/000:04:290
268077999210,1cyclictest0-21swapper/022:13:100
268077999210,19cyclictest3056-21PK-Backend23:55:300
139099210,15rtkit-daemon3056-21PK-Backend21:14:043
139099210,15rtkit-daemon3056-21PK-Backend00:23:443
139099210,14rtkit-daemon3056-21PK-Backend23:04:113
139099210,14rtkit-daemon0-21swapper/223:23:522
2680789992018,1cyclictest0-21swapper/322:04:153
2680789992018,1cyclictest0-21swapper/322:04:143
2680789992018,1cyclictest0-21swapper/321:49:143
2680789992018,1cyclictest0-21swapper/321:36:503
2680789992018,1cyclictest0-21swapper/321:26:293
2680789992018,1cyclictest0-21swapper/320:57:363
2680789992018,1cyclictest0-21swapper/320:33:583
2680789992018,1cyclictest0-21swapper/320:22:073
2680789992012,6cyclictest436-20systemd-journal22:25:003
268078999200,19cyclictest3056-21PK-Backend21:56:443
268078999200,19cyclictest3056-21PK-Backend00:14:543
268078999200,19cyclictest0-21swapper/321:43:323
268078999200,19cyclictest0-21swapper/320:05:523
268078999200,19cyclictest0-21swapper/320:04:363
268078999200,18cyclictest3056-21PK-Backend23:38:173
268078999200,18cyclictest3056-21PK-Backend21:31:103
268078599209,7cyclictest978-21snmpd23:50:192
2680785992018,1cyclictest0-21swapper/222:28:532
2680785992017,2cyclictest3056-21PK-Backend22:43:192
2680785992012,6cyclictest2687367-21ntpq19:25:172
268078599200,1cyclictest0-21swapper/222:45:222
268078599200,1cyclictest0-21swapper/220:57:202
268078599200,19cyclictest0-21swapper/223:03:072
268078599200,19cyclictest0-21swapper/222:13:382
268078599200,19cyclictest0-21swapper/221:45:002
268078599200,19cyclictest0-21swapper/221:15:182
268078599200,19cyclictest0-21swapper/221:14:172
268078599200,19cyclictest0-21swapper/220:33:412
268078599200,19cyclictest0-21swapper/220:26:452
268078599200,19cyclictest0-21swapper/220:10:092
268078599200,19cyclictest0-21swapper/219:46:232
268078599200,19cyclictest0-21swapper/200:22:222
268078599200,19cyclictest0-21swapper/200:05:462
2680782992018,1cyclictest3056-21PK-Backend21:37:111
2680782992018,1cyclictest3056-21PK-Backend21:02:031
2680782992018,1cyclictest3056-21PK-Backend21:02:031
2680782992018,1cyclictest3056-21PK-Backend20:51:161
2680782992018,1cyclictest3056-21PK-Backend20:32:081
2680782992018,1cyclictest3056-21PK-Backend20:07:481
2680782992018,1cyclictest3056-21PK-Backend19:55:381
2680782992018,1cyclictest3056-21PK-Backend19:23:341
2680782992018,1cyclictest0-21swapper/123:06:071
2680782992018,1cyclictest0-21swapper/122:52:281
2680782992017,2cyclictest3056-21PK-Backend22:14:441
2680782992017,2cyclictest3056-21PK-Backend20:56:081
2680782992011,7cyclictest2730906-21needreboot21:30:171
268078299200,19cyclictest3056-21PK-Backend22:47:351
268078299200,19cyclictest3056-21PK-Backend21:22:581
268078299200,19cyclictest3056-21PK-Backend00:33:471
268078299200,19cyclictest0-21swapper/123:24:051
268078299200,19cyclictest0-21swapper/123:01:131
268078299200,19cyclictest0-21swapper/121:53:011
268078299200,19cyclictest0-21swapper/100:12:311
2680779992018,1cyclictest0-21swapper/023:44:150
2680779992018,1cyclictest0-21swapper/023:11:440
2680779992018,1cyclictest0-21swapper/023:02:580
2680779992018,1cyclictest0-21swapper/021:28:430
2680779992018,1cyclictest0-21swapper/020:17:030
2680779992018,1cyclictest0-21swapper/019:34:450
2680779992012,7cyclictest2790102-21fw_packets00:20:150
2680779992012,6cyclictest2774186-21expr23:35:140
2680779992012,5cyclictest436-20systemd-journal22:40:000
268077999200,19cyclictest0-21swapper/023:30:090
268077999200,19cyclictest0-21swapper/022:58:050
268077999200,19cyclictest0-21swapper/022:45:270
268077999200,19cyclictest0-21swapper/022:17:020
268077999200,19cyclictest0-21swapper/021:04:570
268077999200,19cyclictest0-21swapper/021:04:560
268077999200,19cyclictest0-21swapper/019:14:170
27516632190,4sleep13056-21PK-Backend22:30:131
2680789991917,1cyclictest3056-21PK-Backend00:29:003
2680789991917,1cyclictest0-21swapper/322:29:493
2680789991917,1cyclictest0-21swapper/322:15:503
2680789991917,1cyclictest0-21swapper/322:11:583
2680789991917,1cyclictest0-21swapper/300:31:033
2680789991913,5cyclictest2785662-21sshd00:06:143
2680789991913,4cyclictest42-21ksoftirqd/300:35:213
2680789991911,6cyclictest2709188-21munin-plugin-st20:30:003
268078999190,6cyclictest2713222-21expr20:40:123
268078999190,1cyclictest3056-21PK-Backend23:24:113
268078999190,1cyclictest3056-21PK-Backend21:52:043
268078999190,18cyclictest0-21swapper/320:17:143
2680785991917,1cyclictest3056-21PK-Backend21:50:402
2680785991917,1cyclictest0-21swapper/222:38:262
2680785991917,1cyclictest0-21swapper/222:18:322
2680785991917,1cyclictest0-21swapper/221:35:262
2680785991917,1cyclictest0-21swapper/220:42:272
2680785991917,1cyclictest0-21swapper/219:35:292
2680785991917,1cyclictest0-21swapper/200:00:392
2680785991915,3cyclictest978-21snmpd20:07:022
2680785991915,3cyclictest978-21snmpd19:52:132
2680785991911,6cyclictest34-21ksoftirqd/219:55:122
2680785991911,6cyclictest2682717-21ntpq19:15:022
268078599191,13cyclictest2720931-21runrttasks21:00:232
268078599191,13cyclictest2720931-21runrttasks21:00:232
268078599190,1cyclictest0-21swapper/223:29:442
268078599190,1cyclictest0-21swapper/222:03:042
268078599190,1cyclictest0-21swapper/222:03:042
268078599190,1cyclictest0-21swapper/221:47:502
268078599190,1cyclictest0-21swapper/221:30:332
268078599190,1cyclictest0-21swapper/221:06:342
268078599190,18cyclictest0-21swapper/223:46:192
268078599190,18cyclictest0-21swapper/220:37:342
268078599190,18cyclictest0-21swapper/219:21:082
268078299193,10cyclictest2765162-21apt23:10:121
2680782991917,1cyclictest3056-21PK-Backend22:17:331
2680782991917,1cyclictest3056-21PK-Backend22:09:021
2680782991917,1cyclictest3056-21PK-Backend20:47:231
2680782991917,1cyclictest3056-21PK-Backend20:02:141
2680782991917,1cyclictest3056-21PK-Backend19:37:201
2680782991916,2cyclictest3056-21PK-Backend23:48:251
268078299190,1cyclictest3056-21PK-Backend22:01:311
268078299190,1cyclictest3056-21PK-Backend22:01:311
268078299190,1cyclictest3056-21PK-Backend20:40:401
268078299190,1cyclictest3056-21PK-Backend19:41:391
268078299190,1cyclictest0-21swapper/123:18:581
268078299190,1cyclictest0-21swapper/122:56:201
268078299190,18cyclictest3056-21PK-Backend21:40:021
2680779991917,1cyclictest0-21swapper/022:05:390
2680779991917,1cyclictest0-21swapper/021:54:170
2680779991917,1cyclictest0-21swapper/020:45:090
2680779991917,1cyclictest0-21swapper/019:52:170
2680779991917,1cyclictest0-21swapper/019:48:580
2680779991912,5cyclictest436-20systemd-journal00:15:010
2680779991912,5cyclictest2708546-21cut20:25:210
268077999190,1cyclictest0-21swapper/022:50:210
268077999190,1cyclictest0-21swapper/021:09:020
268077999190,1cyclictest0-21swapper/020:43:060
268077999190,18cyclictest0-21swapper/000:29:030
139099190,10rtkit-daemon2743493-21python322:05:222
268078999181,4cyclictest2722334-21/usr/sbin/munin21:05:203
2680789991814,3cyclictest2687382-21ntp_states19:25:193
268078999180,1cyclictest0-21swapper/319:13:263
268078599184,12cyclictest2768361-21latency_hist23:20:002
2680785991816,1cyclictest0-21swapper/223:57:352
2680785991816,1cyclictest0-21swapper/221:57:232
2680785991816,1cyclictest0-21swapper/219:42:032
2680785991816,1cyclictest0-21swapper/200:16:142
2680785991813,4cyclictest2682410-21cron19:15:002
2680785991812,5cyclictest2794182-21ntpq00:30:182
2680785991810,6cyclictest2748576-21ntp_states22:20:202
2680785991810,6cyclictest2717605-21sshd20:52:132
2680785991810,6cyclictest2699895-21sensors_temp20:00:202
268078599180,1cyclictest0-21swapper/220:47:202
2680779991816,1cyclictest0-21swapper/022:29:120
2680779991813,3cyclictest2784651-21cpuspeed_turbos00:05:140
2680789991715,1cyclictest0-21swapper/321:21:493
2680789991715,1cyclictest0-21swapper/319:44:363
2680789991712,3cyclictest2777925-21df_inode23:45:143
268078599179,6cyclictest2752266-21sed22:30:222
2680785991711,5cyclictest2729115-21sed21:25:182
2680785991711,4cyclictest2772604-21cut23:30:162
2680782991713,2cyclictest26-21ksoftirqd/122:30:001
2680779991715,1cyclictest0-21swapper/021:11:040
2680779991712,3cyclictest2702233-21sh20:10:000
268078599169,6cyclictest2786705-21cat00:10:162
2680785991614,1cyclictest0-21swapper/200:35:542
2680785991613,2cyclictest1095-21runrttasks21:21:072
268078299169,5cyclictest2775605-21chmod23:40:011
2680779991614,1cyclictest0-21swapper/021:24:020
2680779991612,3cyclictest2688114-21latency_hist19:30:010
268077999161,12cyclictest2787974-21apt00:15:110
268078999150,13cyclictest2779258-21taskset23:50:083
2680785991513,1cyclictest0-21swapper/222:52:182
268078299150,5cyclictest3056-21PK-Backend00:06:321
2680779991513,1cyclictest0-21swapper/021:47:480
2680785991410,3cyclictest2791869-21cpuspeed_turbos00:25:122
268078299130,6cyclictest3056-21PK-Backend19:15:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional