You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Jul 27, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283890921410,3sleep20-21swapper/207:25:212
293024721240,6sleep342-21ksoftirqd/311:45:213
284585321200,8sleep3411rcuc/307:45:233
139099800,65rtkit-daemon0-21swapper/107:05:101
283187227739,14sleep30-21swapper/307:08:093
283169426545,14sleep00-21swapper/007:05:490
283178326141,14sleep20-21swapper/207:07:012
28566772420,4sleep22856674-21netstat08:15:162
283215499300,18cyclictest0-21swapper/311:37:563
2832154992913,14cyclictest2832804-21cat07:10:153
283215499290,18cyclictest0-21swapper/312:06:313
28743122280,4sleep13056-21PK-Backend09:05:211
2832154992813,13cyclictest2871793-21sed09:00:013
283215499280,16cyclictest0-21swapper/311:33:593
2832154992712,13cyclictest2844490-21seq07:41:383
2832143992715,7cyclictest1095-21runrttasks08:10:040
28829892260,4chrt3056-21PK-Backend09:30:210
28586282260,4sleep1251rcuc/108:20:171
283215499268,16cyclictest2869263-21cat08:50:213
283215499266,18cyclictest2866399-21unin-run08:44:593
2832154992614,10cyclictest2927969-21hddtemp_smartct11:40:173
2832154992614,10cyclictest2927969-21hddtemp_smartct11:40:163
283215499261,23cyclictest2932169-21users11:50:243
283215499260,15cyclictest0-21swapper/308:49:383
283215499260,14cyclictest0-21swapper/309:18:013
28460892250,2sleep10-21swapper/107:45:241
283215499258,15cyclictest2900296-21ntp_states10:20:183
283215499252,18cyclictest2863174-21apt08:35:113
283215499250,6cyclictest2921492-21python311:20:223
2832147992511,12cyclictest2869865-21unin-run08:55:001
28836892240,7chrt2883684-21cron09:35:001
283215499241,21cyclictest978-21snmpd08:22:353
2832154992410,12cyclictest2839510-21dump-pmu-power07:30:013
283215499240,9cyclictest2873442-21cat09:05:003
283215499240,22cyclictest2156-21gnome-shell11:11:593
283215499237,14cyclictest2906641-21wc10:40:003
283215499237,13cyclictest2898707-21switchtime10:15:223
283215499235,17cyclictest2862291-21gzip08:30:203
283215499234,17cyclictest2916180-21awk11:05:203
283215499234,17cyclictest2847832-21sed07:50:213
283215499232,18cyclictest0-21swapper/310:10:133
283215499231,20cyclictest2854945-21ntpq08:10:193
283215499230,21cyclictest0-21swapper/309:40:123
283215499230,20cyclictest2891633-21taskset09:55:193
2832143992314,6cyclictest883-21dbus-daemon08:20:010
283214399230,21cyclictest3056-21PK-Backend07:40:020
29212602220,3chrt0-21swapper/211:20:182
283215499229,12cyclictest28610332chrt08:28:303
283215499229,11cyclictest2936027-21latency_hist12:05:013
283215499229,11cyclictest2910588-21sh10:50:143
283215499229,11cyclictest2904678-21anacron10:34:133
283215499228,12cyclictest2840842-21sed07:30:223
283215499224,16cyclictest2919035-21expr11:15:143
283215499224,16cyclictest2849370-21swap07:55:213
283215499223,17cyclictest2945410-21expr12:30:133
283215499223,16cyclictest2948101-21/usr/sbin/munin12:35:233
283215499222,18cyclictest2888094-21tail09:45:193
283215499222,18cyclictest2856185-21cstates08:15:123
283215499222,15cyclictest2836317-21apt07:20:123
2832154992217,3cyclictest2938605-21memory12:10:183
283215499221,19cyclictest2922421-21sort11:25:133
283215499221,19cyclictest2913550-21latency_hist11:00:003
283215499221,19cyclictest2881365-21python309:25:223
283215499221,19cyclictest2875190-21latency_hist09:10:003
283215499221,18cyclictest2944090-21python312:25:213
283215499221,18cyclictest0-21swapper/311:55:153
2832154992211,10cyclictest978-21snmpd07:37:273
283215499220,20cyclictest2941495-21apt12:20:113
283215499220,20cyclictest2893004-21taskset10:00:143
2832149992218,2cyclictest2945973-21ntp_states12:30:192
2832149992215,6cyclictest2833097-21ntp_states07:10:192
283214999220,21cyclictest2889136-21/usr/sbin/munin09:50:192
2832143992220,1cyclictest3056-21PK-Backend12:30:350
2832143992220,1cyclictest0-21swapper/012:15:330
2832143992220,1cyclictest0-21swapper/011:06:260
2832143992213,6cyclictest2901031-21apt-get10:25:000
283214399220,21cyclictest0-21swapper/007:46:450
283214399220,21cyclictest0-21swapper/007:24:210
283214399220,21cyclictest0-21swapper/007:16:230
139099220,3rtkit-daemon0-21swapper/007:26:290
28803992210,4sleep10-21swapper/109:25:011
283215499217,12cyclictest2890259-21sshd09:50:293
283215499215,14cyclictest2914094-21sh11:00:143
283215499214,15cyclictest2940222-21cat12:15:183
283215499214,15cyclictest2909193-21ntpq10:45:193
283215499214,15cyclictest2907365-21missed_timers10:40:193
283215499214,15cyclictest2907365-21missed_timers10:40:183
283215499213,16cyclictest2882360-21cpuspeed_turbos09:30:143
2832154992112,8cyclictest42-21ksoftirqd/308:05:193
283215499211,18cyclictest1-21systemd07:15:013
283215499211,15cyclictest2894362-21apt10:05:123
283215499210,19cyclictest2903036-21turbostat10:30:003
283215499210,19cyclictest2878903-21grep09:20:133
283215499210,19cyclictest2851736-21apt-get08:05:003
283215499210,18cyclictest2875676-21diskstats09:10:153
2832149992119,1cyclictest0-21swapper/210:35:072
2832149992115,4cyclictest2156-21gnome-shell12:20:092
2832149992113,6cyclictest2868104-21cron08:49:592
2832143992119,1cyclictest3056-21PK-Backend11:16:390
2832143992119,1cyclictest0-21swapper/011:42:550
2832143992119,1cyclictest0-21swapper/011:42:550
2832143992119,1cyclictest0-21swapper/011:13:140
2832143992119,1cyclictest0-21swapper/010:53:340
2832143992119,1cyclictest0-21swapper/010:10:500
2832143992119,1cyclictest0-21swapper/009:49:200
2832143992119,1cyclictest0-21swapper/009:27:290
2832143992118,2cyclictest3056-21PK-Backend11:45:320
2832143992118,2cyclictest3056-21PK-Backend09:16:560
2832143992118,2cyclictest0-21swapper/011:58:110
283214399210,20cyclictest3056-21PK-Backend12:25:350
283214399210,20cyclictest0-21swapper/011:20:040
283214399210,20cyclictest0-21swapper/010:35:290
283214399210,20cyclictest0-21swapper/010:08:120
283214399210,20cyclictest0-21swapper/009:41:430
139099210,6rtkit-daemon3056-21PK-Backend09:28:181
283215499202,16cyclictest2884573-21missed_timers09:35:163
2832149992018,1cyclictest0-21swapper/210:54:142
2832149992018,1cyclictest0-21swapper/210:28:532
2832149992018,1cyclictest0-21swapper/210:21:152
2832149992018,1cyclictest0-21swapper/209:40:522
2832149992018,1cyclictest0-21swapper/209:30:392
2832149992013,6cyclictest2836979-21grep07:20:172
2832149992013,5cyclictest2850228-21cat08:00:112
2832149992013,5cyclictest2835284-21ntp_kernel_err07:15:192
2832149992012,6cyclictest2843749-21cat07:40:152
283214999200,19cyclictest0-21swapper/212:25:182
283214999200,19cyclictest0-21swapper/212:15:192
283214999200,19cyclictest0-21swapper/212:10:192
283214999200,19cyclictest0-21swapper/211:08:532
283214999200,19cyclictest0-21swapper/211:02:522
283214999200,19cyclictest0-21swapper/210:56:502
283214999200,19cyclictest0-21swapper/210:30:422
283214999200,19cyclictest0-21swapper/208:42:462
283214999200,19cyclictest0-21swapper/208:10:352
2832147992013,6cyclictest2851307-21awk08:00:231
283214799200,1cyclictest0-21swapper/111:08:211
283214799200,19cyclictest3056-21PK-Backend09:46:221
283214799200,19cyclictest0-21swapper/111:40:191
283214799200,19cyclictest0-21swapper/111:40:191
283214799200,19cyclictest0-21swapper/108:57:351
2832143992018,1cyclictest3056-21PK-Backend11:53:570
2832143992018,1cyclictest3056-21PK-Backend11:03:010
2832143992018,1cyclictest0-21swapper/008:42:270
2832143992018,1cyclictest0-21swapper/007:13:040
283214399200,1cyclictest3056-21PK-Backend10:01:220
283214399200,19cyclictest3056-21PK-Backend07:37:040
283214399200,19cyclictest0-21swapper/011:31:540
283214399200,19cyclictest0-21swapper/010:47:320
283214399200,19cyclictest0-21swapper/008:26:460
283214399200,19cyclictest0-21swapper/008:01:580
283214399200,19cyclictest0-21swapper/007:55:500
283214399200,19cyclictest0-21swapper/007:31:570
283214399200,18cyclictest2859045-21awk08:20:220
2832149991917,1cyclictest3056-21PK-Backend09:47:422
2832149991917,1cyclictest3056-21PK-Backend09:19:022
2832149991917,1cyclictest0-21swapper/212:39:182
2832149991917,1cyclictest0-21swapper/211:50:512
2832149991917,1cyclictest0-21swapper/211:26:102
2832149991917,1cyclictest0-21swapper/209:38:152
2832149991917,1cyclictest0-21swapper/208:05:142
2832149991916,2cyclictest0-21swapper/212:05:192
2832149991912,5cyclictest2907073-21expr10:40:132
2832149991912,5cyclictest2907073-21expr10:40:132
2832149991912,5cyclictest2862912-21munin-run08:35:012
2832149991911,6cyclictest2935065-21cut12:00:162
283214999190,1cyclictest0-21swapper/211:38:002
283214999190,1cyclictest0-21swapper/211:17:332
283214999190,1cyclictest0-21swapper/211:11:312
283214999190,18cyclictest3056-21PK-Backend10:02:232
283214999190,18cyclictest0-21swapper/211:59:162
283214999190,18cyclictest0-21swapper/210:49:002
283214999190,18cyclictest0-21swapper/209:12:272
283214999190,18cyclictest0-21swapper/208:56:532
283214999190,18cyclictest0-21swapper/207:36:562
2832147991917,1cyclictest3056-21PK-Backend07:30:351
2832147991917,1cyclictest0-21swapper/111:54:191
2832147991917,1cyclictest0-21swapper/110:14:011
2832147991917,1cyclictest0-21swapper/108:38:501
2832147991911,6cyclictest2947140-21sed12:35:001
283214799190,1cyclictest3056-21PK-Backend10:41:181
283214799190,1cyclictest3056-21PK-Backend10:41:171
283214799190,1cyclictest0-21swapper/111:36:331
283214799190,1cyclictest0-21swapper/110:29:011
283214799190,1cyclictest0-21swapper/107:57:461
2832143991917,1cyclictest3056-21PK-Backend09:03:430
2832143991917,1cyclictest0-21swapper/010:44:090
2832143991917,1cyclictest0-21swapper/010:44:090
2832143991917,1cyclictest0-21swapper/008:05:090
2832143991916,2cyclictest3056-21PK-Backend09:37:290
2832143991914,3cyclictest1895009-21snapd09:08:430
283214399190,1cyclictest3056-21PK-Backend10:56:590
283214399190,1cyclictest3056-21PK-Backend08:39:160
283214399190,1cyclictest0-21swapper/008:57:560
283214399190,1cyclictest0-21swapper/007:54:150
283214999189,7cyclictest883-21dbus-daemon07:32:352
283214999188,5cyclictest2925696-21cut11:35:002
2832149991816,1cyclictest0-21swapper/209:55:342
2832149991816,1cyclictest0-21swapper/208:22:182
2832149991814,3cyclictest2896097-21apt10:10:132
2832149991811,6cyclictest2895856-21systemd-run10:10:012
2832149991810,6cyclictest2874542-21sed09:05:242
2832149991810,6cyclictest2847392-21sed07:50:172
283214999180,6cyclictest3056-21PK-Backend08:54:592
283214999180,1cyclictest0-21swapper/211:40:382
283214999180,1cyclictest0-21swapper/211:40:382
2832147991816,1cyclictest3056-21PK-Backend08:43:441
2832147991816,1cyclictest3056-21PK-Backend07:16:371
2832147991816,1cyclictest0-21swapper/112:05:201
2832147991816,1cyclictest0-21swapper/109:39:401
2832147991816,1cyclictest0-21swapper/108:12:271
2832147991814,3cyclictest2857486-21apt-config08:20:001
2832147991811,5cyclictest2911173-21grep10:50:231
2832147991811,5cyclictest2838603-21cat07:25:161
283214799180,1cyclictest0-21swapper/107:20:061
283214399183,9cyclictest2901031-21apt-get10:25:030
2832143991816,1cyclictest0-21swapper/011:27:420
2832143991816,1cyclictest0-21swapper/009:51:090
2832143991811,4cyclictest1095-21runrttasks11:35:140
283214399181,12cyclictest2946685-21apt-get12:35:030
283214399180,1cyclictest0-21swapper/008:48:370
283214399180,16cyclictest2876751-21cron09:15:000
2832149991711,5cyclictest2880681-21expr09:25:132
2832149991711,4cyclictest1095-21runrttasks10:17:552
2832149991710,5cyclictest2879515-21ntpq09:20:192
2832149991710,5cyclictest2174-21JS10:27:482
283214799179,6cyclictest436-20systemd-journal12:15:001
283214799173,5cyclictest26-21ksoftirqd/112:30:191
283214799173,2cyclictest141rcu_preempt11:34:591
2832147991715,1cyclictest0-21swapper/111:27:421
2832147991712,3cyclictest2853793-21gpgconf08:10:011
2832147991712,3cyclictest2833823-21latency_hist07:15:011
2832147991711,4cyclictest2944924-21date12:30:021
2832147991711,4cyclictest2871993-21tr09:00:141
2832147991710,5cyclictest2934717-21cpuspeed_turbos12:00:121
2832147991710,5cyclictest2848391-21grep07:55:001
283214399177,8cyclictest883-21dbus-daemon08:55:010
2832143991715,1cyclictest3056-21PK-Backend12:04:460
2832143991715,1cyclictest3056-21PK-Backend08:31:320
2832143991715,1cyclictest0-21swapper/010:32:520
2832143991715,1cyclictest0-21swapper/010:19:150
2832143991714,2cyclictest2942948-21sh12:25:000
2832143991712,3cyclictest2891032-21cpuspeed_turbos09:55:120
283214999168,6cyclictest2845971-21sed07:45:232
2832149991614,1cyclictest0-21swapper/208:25:302
2832149991611,3cyclictest2930019-21ntp_kernel_pll_11:45:192
283214799168,6cyclictest2930009-21awk11:45:191
283214799163,2cyclictest141rcu_preempt08:50:001
2832147991614,1cyclictest3056-21PK-Backend11:18:491
2832147991614,1cyclictest0-21swapper/111:00:171
283214799161,2cyclictest3056-21PK-Backend11:10:181
2832147991612,3cyclictest2912378-21idleruntime10:55:181
2832147991612,3cyclictest2892359-21sh10:00:001
2832147991611,4cyclictest3056-21PK-Backend10:22:311
2832147991610,4cyclictest883-21dbus-daemon09:15:011
2832147991610,4cyclictest2844189-21grep07:40:221
283214399167,7cyclictest2938026-21apt12:10:130
2832143991614,1cyclictest3056-21PK-Backend12:09:450
283214999159,5cyclictest2864115-21find08:35:222
283214999157,6cyclictest2872099-21expr09:00:152
283214799159,5cyclictest2933382-21sed11:55:201
283214799159,5cyclictest26-21ksoftirqd/108:30:001
283214799157,6cyclictest2897817-21awk10:15:101
2832147991513,1cyclictest0-21swapper/110:05:091
2832147991511,3cyclictest2941891-21hddtemp_smartct12:20:161
2832147991511,3cyclictest2893893-21sshd10:02:371
2832147991511,3cyclictest2841672-21cpuspeed_turbos07:35:141
2832147991510,3cyclictest28903272sleep109:51:201
2832147991510,3cyclictest28626862sleep108:32:251
283214799150,1cyclictest3056-21PK-Backend10:35:581
2832147991412,1cyclictest3056-21PK-Backend10:31:251
2832147991411,2cyclictest3056-21PK-Backend09:15:061
283214799140,13cyclictest3056-21PK-Backend12:16:571
283214799140,13cyclictest3056-21PK-Backend11:21:471
2832143991410,3cyclictest2878774-21cpu09:20:130
283214799138,3cyclictest28856722sleep109:40:101
283214799131,10cyclictest2908861-21diskstats10:45:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional