You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 05:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Sun Aug 02, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
174345321360,9sleep00-21swapper/019:10:000
185121121170,4sleep13056-21PK-Backend00:15:001
174877921020,3sleep00-21swapper/019:20:230
174310028564,14sleep20-21swapper/219:06:062
174312127538,31sleep30-21swapper/319:06:233
174324026545,14sleep10-21swapper/119:07:531
18153472580,3sleep10-21swapper/122:30:191
17991272510,3sleep30-21swapper/321:45:143
139099500,3rtkit-daemon0-21swapper/220:29:443
139099490,3rtkit-daemon0-21swapper/222:24:203
139099370,3rtkit-daemon0-21swapper/222:31:362
17926612290,4sleep13056-21PK-Backend21:25:191
18069312270,2sleep20-21swapper/222:05:212
1743562992716,3cyclictest26-21ksoftirqd/100:08:061
174356299260,7cyclictest3056-21PK-Backend23:45:001
174356299260,7cyclictest3056-21PK-Backend23:45:001
1743562992522,2cyclictest251rcuc/123:57:061
174356299250,6cyclictest3056-21PK-Backend21:05:011
1743562992421,2cyclictest251rcuc/122:49:321
1743559992416,6cyclictest1788082-21grep21:15:000
174355999240,22cyclictest1749953-21wc19:25:160
139099240,4rtkit-daemon1389-21rtkit-daemon22:54:520
174356499236,3cyclictest141rcu_preempt23:20:162
174356299237,7cyclictest1747674-21seq19:20:071
174356299235,8cyclictest26-21ksoftirqd/122:59:461
1743562992320,2cyclictest3056-21PK-Backend23:09:281
1743562992319,3cyclictest251rcuc/120:10:011
1743559992321,1cyclictest0-21swapper/023:33:580
1743559992320,2cyclictest0-21swapper/023:27:350
139099230,4rtkit-daemon1389-21rtkit-daemon00:38:551
174356899220,3cyclictest1794872-21taskset21:32:493
1743564992220,1cyclictest0-21swapper/222:35:572
1743564992220,1cyclictest0-21swapper/222:35:572
1743564992217,3cyclictest34-21ksoftirqd/223:15:142
1743564992215,5cyclictest1798276-21sshd21:42:032
1743564992214,6cyclictest1808114-21fw_packets22:10:142
174356499220,21cyclictest0-21swapper/222:57:342
174356499220,21cyclictest0-21swapper/221:53:062
174356499220,21cyclictest0-21swapper/200:33:392
174356499220,20cyclictest1827365-21load23:05:172
174356299225,10cyclictest26-21ksoftirqd/120:50:161
1743562992220,1cyclictest0-21swapper/123:53:291
1743562992220,1cyclictest0-21swapper/120:38:281
1743562992219,2cyclictest3056-21PK-Backend19:46:091
1743562992219,2cyclictest3056-21PK-Backend19:46:091
1743562992219,2cyclictest3056-21PK-Backend00:24:111
174356299221,8cyclictest26-21ksoftirqd/122:15:171
1743562992218,3cyclictest251rcuc/122:45:001
1743562992218,3cyclictest251rcuc/122:45:001
174356299220,20cyclictest3056-21PK-Backend19:37:381
1743559992220,1cyclictest0-21swapper/023:04:040
1743559992220,1cyclictest0-21swapper/021:17:310
174355999220,3cyclictest1776543-21memory20:40:190
174355999220,21cyclictest0-21swapper/000:02:100
174355999220,20cyclictest3056-21PK-Backend21:23:040
174355999220,20cyclictest0-21swapper/020:15:520
174355999220,20cyclictest0-21swapper/000:34:140
1743568992118,2cyclictest3056-21PK-Backend00:32:363
1743568992114,5cyclictest1791091-21sed21:20:223
1743568992112,7cyclictest883-21dbus-daemon00:00:003
174356899210,20cyclictest3056-21PK-Backend22:37:593
174356899210,20cyclictest3056-21PK-Backend22:37:593
174356899210,20cyclictest3056-21PK-Backend20:49:503
174356899210,20cyclictest3056-21PK-Backend20:49:503
174356899210,19cyclictest3056-21PK-Backend22:56:503
1743564992119,1cyclictest0-21swapper/223:58:032
1743564992119,1cyclictest0-21swapper/220:05:112
1743564992119,1cyclictest0-21swapper/200:35:022
1743564992119,1cyclictest0-21swapper/200:05:142
1743564992113,7cyclictest1763284-21apt-get20:05:002
1743564992113,5cyclictest1095-21runrttasks21:35:122
174356499210,20cyclictest0-21swapper/220:46:212
174356499210,20cyclictest0-21swapper/220:46:212
174356499210,20cyclictest0-21swapper/219:43:412
174356499210,20cyclictest0-21swapper/200:03:512
174356499210,1cyclictest0-21swapper/219:51:392
174356499210,1cyclictest0-21swapper/200:16:502
174356499210,19cyclictest3056-21PK-Backend23:35:252
174356299218,4cyclictest26-21ksoftirqd/119:15:001
174356299213,6cyclictest26-21ksoftirqd/120:40:231
174356299212,4cyclictest26-21ksoftirqd/121:33:001
1743562992119,1cyclictest3056-21PK-Backend23:33:011
1743562992119,1cyclictest3056-21PK-Backend21:13:291
1743562992119,1cyclictest3056-21PK-Backend20:59:301
1743562992119,1cyclictest0-21swapper/123:15:501
1743562992119,1cyclictest0-21swapper/122:35:221
1743562992119,1cyclictest0-21swapper/122:35:221
1743562992119,1cyclictest0-21swapper/122:26:431
1743562992119,1cyclictest0-21swapper/120:29:491
1743562992119,1cyclictest0-21swapper/120:12:251
1743562992118,2cyclictest251rcuc/119:58:311
1743562992116,4cyclictest26-21ksoftirqd/120:35:011
1743562992114,6cyclictest1749803-21df_inode19:25:121
1743562992112,3cyclictest26-21ksoftirqd/100:02:291
174356299210,19cyclictest3056-21PK-Backend21:39:391
174356299210,19cyclictest18002312sleep121:49:521
1743559992119,1cyclictest3056-21PK-Backend23:41:420
1743559992119,1cyclictest3056-21PK-Backend23:41:420
1743559992119,1cyclictest0-21swapper/023:53:510
1743559992119,1cyclictest0-21swapper/023:16:470
1743559992119,1cyclictest0-21swapper/022:36:190
1743559992119,1cyclictest0-21swapper/022:36:190
1743559992119,1cyclictest0-21swapper/022:30:450
1743559992119,1cyclictest0-21swapper/021:59:080
1743559992119,1cyclictest0-21swapper/020:54:520
1743559992119,1cyclictest0-21swapper/020:31:190
1743559992119,1cyclictest0-21swapper/020:25:390
1743559992119,1cyclictest0-21swapper/020:23:300
1743559992119,1cyclictest0-21swapper/019:55:380
1743559992119,1cyclictest0-21swapper/019:45:170
1743559992119,1cyclictest0-21swapper/019:45:170
1743559992119,1cyclictest0-21swapper/019:42:540
1743559992119,1cyclictest0-21swapper/019:35:570
1743559992115,4cyclictest1844240-21sort23:55:140
174355999210,20cyclictest3056-21PK-Backend21:41:230
174355999210,20cyclictest0-21swapper/021:50:150
174355999210,20cyclictest0-21swapper/020:59:040
174355999210,20cyclictest0-21swapper/019:13:200
17883522200,4sleep13056-21PK-Backend21:15:121
1743568992018,1cyclictest3056-21PK-Backend23:48:563
1743568992018,1cyclictest3056-21PK-Backend19:56:233
1743568992018,1cyclictest0-21swapper/323:35:383
1743568992018,1cyclictest0-21swapper/321:12:143
1743568992017,2cyclictest3056-21PK-Backend22:42:593
1743568992017,2cyclictest3056-21PK-Backend22:42:593
1743568992017,2cyclictest3056-21PK-Backend22:31:493
1743568992014,4cyclictest2156-21gnome-shell21:59:083
174356899201,17cyclictest1784860-21/usr/sbin/munin21:05:253
1743568992010,8cyclictest42-21ksoftirqd/323:50:153
174356899200,1cyclictest3056-21PK-Backend22:01:343
174356899200,19cyclictest3056-21PK-Backend20:18:343
174356899200,19cyclictest3056-21PK-Backend20:04:553
174356899200,19cyclictest3056-21PK-Backend19:27:373
174356899200,19cyclictest0-21swapper/321:03:353
1743564992019,1cyclictest0-21swapper/222:42:542
1743564992019,1cyclictest0-21swapper/222:42:542
1743564992018,1cyclictest0-21swapper/223:46:282
1743564992018,1cyclictest0-21swapper/223:31:342
1743564992018,1cyclictest0-21swapper/221:49:142
1743564992018,1cyclictest0-21swapper/219:25:162
1743564992018,1cyclictest0-21swapper/200:12:582
1743564992014,3cyclictest1854532-21turbostat.cron00:25:002
174356499200,1cyclictest0-21swapper/221:30:062
174356499200,1cyclictest0-21swapper/219:11:102
174356499200,19cyclictest3056-21PK-Backend20:25:182
174356499200,19cyclictest0-21swapper/222:47:202
174356499200,19cyclictest0-21swapper/221:25:412
174356499200,19cyclictest0-21swapper/220:23:092
174356499200,19cyclictest0-21swapper/219:46:522
174356499200,19cyclictest0-21swapper/219:46:512
174356299203,7cyclictest26-21ksoftirqd/100:15:571
1743562992018,1cyclictest3056-21PK-Backend21:58:451
1743562992018,1cyclictest3056-21PK-Backend21:52:231
1743562992018,1cyclictest3056-21PK-Backend20:45:101
1743562992018,1cyclictest3056-21PK-Backend20:45:091
1743562992018,1cyclictest0-21swapper/123:11:241
1743562992018,1cyclictest0-21swapper/121:06:471
174356299201,7cyclictest26-21ksoftirqd/123:35:201
174356299201,7cyclictest26-21ksoftirqd/122:20:221
174356299200,6cyclictest1807451-21sh22:10:001
174356299200,5cyclictest1747425-21cron19:20:001
174356299200,19cyclictest3056-21PK-Backend23:00:371
174356299200,19cyclictest0-21swapper/100:30:331
1743559992018,1cyclictest0-21swapper/023:12:220
1743559992018,1cyclictest0-21swapper/022:41:190
1743559992018,1cyclictest0-21swapper/022:41:190
1743559992018,1cyclictest0-21swapper/021:39:270
1743559992018,1cyclictest0-21swapper/020:48:580
1743559992018,1cyclictest0-21swapper/020:48:580
1743559992018,1cyclictest0-21swapper/019:34:220
1743559992018,1cyclictest0-21swapper/000:35:360
1743559992018,1cyclictest0-21swapper/000:15:410
1743559992016,3cyclictest0-21swapper/022:55:110
1743559992011,7cyclictest1746329-21df19:15:140
174355999200,19cyclictest0-21swapper/023:37:500
174355999200,19cyclictest0-21swapper/023:23:430
174355999200,19cyclictest0-21swapper/022:46:180
174355999200,19cyclictest0-21swapper/021:45:490
174355999200,19cyclictest0-21swapper/021:05:480
174355999200,19cyclictest0-21swapper/021:02:440
174355999200,19cyclictest0-21swapper/020:36:070
1743568991917,1cyclictest3056-21PK-Backend19:14:523
1743568991917,1cyclictest3056-21PK-Backend00:38:583
1743568991917,1cyclictest0-21swapper/322:12:033
1743568991917,1cyclictest0-21swapper/322:06:493
1743568991917,1cyclictest0-21swapper/321:39:113
1743568991917,1cyclictest0-21swapper/320:42:333
1743568991917,1cyclictest0-21swapper/319:32:103
1743568991917,1cyclictest0-21swapper/300:28:443
1743568991915,3cyclictest0-21swapper/320:10:223
1743568991912,5cyclictest1849334-21cron00:10:013
1743568991912,5cyclictest1836591-21sshd23:31:013
1743568991912,5cyclictest1828600-21cat23:10:133
174356899191,16cyclictest1797792-21/usr/sbin/munin21:40:183
174356899191,15cyclictest0-21swapper/322:45:123
174356899190,1cyclictest3056-21PK-Backend23:27:193
174356899190,1cyclictest3056-21PK-Backend19:54:483
174356899190,18cyclictest3056-21PK-Backend22:52:243
174356899190,18cyclictest3056-21PK-Backend20:34:423
174356899190,18cyclictest0-21swapper/323:44:303
174356899190,18cyclictest0-21swapper/323:44:303
174356899190,18cyclictest0-21swapper/323:05:093
174356899190,18cyclictest0-21swapper/321:54:243
174356899190,18cyclictest0-21swapper/300:22:553
174356499197,11cyclictest1823042-21cron22:55:002
174356499192,3cyclictest141rcu_preempt22:25:012
1743564991917,1cyclictest0-21swapper/221:10:402
1743564991917,1cyclictest0-21swapper/221:08:102
1743564991917,1cyclictest0-21swapper/220:10:382
1743564991915,3cyclictest1095-21runrttasks23:03:092
174356499190,1cyclictest0-21swapper/223:43:102
174356499190,1cyclictest0-21swapper/223:43:102
174356499190,1cyclictest0-21swapper/222:29:022
174356499190,1cyclictest0-21swapper/221:02:022
174356499190,1cyclictest0-21swapper/220:58:582
174356499190,1cyclictest0-21swapper/220:54:462
174356499190,1cyclictest0-21swapper/220:34:382
174356499190,1cyclictest0-21swapper/219:35:092
174356499190,1cyclictest0-21swapper/219:33:342
174356499190,1cyclictest0-21swapper/200:25:552
174356299191,8cyclictest26-21ksoftirqd/120:00:251
1743562991917,1cyclictest0-21swapper/121:40:591
1743562991916,1cyclictest3056-21PK-Backend19:40:011
1743562991915,3cyclictest251rcuc/120:17:361
1743562991912,5cyclictest26-21ksoftirqd/123:25:011
174356299190,6cyclictest3056-21PK-Backend21:21:091
174356299190,6cyclictest1833719-21apt23:25:111
174356299190,1cyclictest3056-21PK-Backend19:30:061
174356299190,1cyclictest3056-21PK-Backend00:28:381
174356299190,1cyclictest0-21swapper/120:22:451
174356299190,18cyclictest0-21swapper/122:54:491
1743559991917,1cyclictest0-21swapper/022:11:310
1743559991917,1cyclictest0-21swapper/022:00:300
1743559991917,1cyclictest0-21swapper/020:04:560
1743559991917,1cyclictest0-21swapper/000:11:490
1743559991915,3cyclictest1765727-21tr20:10:160
1743559991913,4cyclictest2156-21gnome-shell23:46:080
174355999190,1cyclictest3056-21PK-Backend23:06:000
174355999190,1cyclictest0-21swapper/021:35:010
174355999190,1cyclictest0-21swapper/000:22:030
1743568991816,1cyclictest3056-21PK-Backend19:24:403
1743568991816,1cyclictest0-21swapper/323:15:233
1743568991815,2cyclictest3056-21PK-Backend20:07:383
1743568991814,3cyclictest1788767-21diskstats21:15:153
1743568991813,3cyclictest436-20systemd-journal00:04:553
1743568991813,3cyclictest1753744-21memory19:35:173
1743568991812,4cyclictest1825397-21expr23:00:133
174356899181,15cyclictest1781160-21apt20:55:113
174356499189,4cyclictest1803774-21cat22:00:012
1743564991816,1cyclictest3056-21PK-Backend19:15:152
1743564991816,1cyclictest0-21swapper/223:52:162
1743564991816,1cyclictest0-21swapper/220:37:082
174356499181,14cyclictest1760097-21apt19:55:122
174356499181,12cyclictest1803721-21apt-get22:00:052
174356299181,7cyclictest26-21ksoftirqd/119:50:131
174356299180,7cyclictest26-21ksoftirqd/123:45:171
174356299180,1cyclictest0-21swapper/122:00:411
1743559991816,1cyclictest0-21swapper/022:24:010
1743559991816,1cyclictest0-21swapper/020:07:400
1743559991816,1cyclictest0-21swapper/019:50:040
1743559991816,1cyclictest0-21swapper/000:27:520
1743559991816,1cyclictest0-21swapper/000:07:580
1743568991715,1cyclictest3056-21PK-Backend20:23:273
1743568991715,1cyclictest0-21swapper/321:27:143
1743568991715,1cyclictest0-21swapper/300:12:423
1743568991713,3cyclictest1756631-21apt19:45:113
1743568991713,3cyclictest1756631-21apt19:45:113
174356899170,5cyclictest1754876-21/usr/sbin/munin19:40:143
174356899170,15cyclictest1852070-21ntp_states00:15:193
174356899170,15cyclictest1810319-21python322:15:213
1743564991716,1cyclictest0-21swapper/220:19:362
1743564991716,1cyclictest0-21swapper/219:21:302
1743564991715,1cyclictest0-21swapper/221:20:412
174356499171,2cyclictest141rcu_preempt22:15:192
1743564991712,3cyclictest34-21ksoftirqd/220:40:232
1743564991711,4cyclictest1788838-21cat21:15:172
174356299170,6cyclictest3056-21PK-Backend22:13:541
1743559991715,1cyclictest0-21swapper/022:27:050
1743559991711,5cyclictest1806176-21tr22:05:130
174356899161,4cyclictest1746641-21head19:15:193
1743568991614,1cyclictest3056-21PK-Backend23:22:203
1743559991611,3cyclictest1792665-21processes21:25:180
1743568991513,1cyclictest3056-21PK-Backend20:38:203
1743568991510,3cyclictest1779864-21if_enp2s020:50:163
174356899150,5cyclictest1813913-21timerandwakeup22:25:223
1743564991511,3cyclictest1833694-21acpi23:25:102
1743564991511,2cyclictest34-21ksoftirqd/223:10:192
174355999159,5cyclictest1810084-21sed22:15:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional