You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Fri Aug 21, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2990328774,9sleep00-21swapper/007:09:070
2968228573,8sleep30-21swapper/307:06:153
2987228365,14sleep10-21swapper/107:08:421
2963928369,10sleep20-21swapper/207:05:422
99712690,1sleep339-21ksoftirqd/309:20:273
4792550,0sleep10-21swapper/109:54:111
30066993717,19cyclictest0-21swapper/209:35:132
30066992723,4cyclictest0-21swapper/210:10:132
309452250,0sleep20-21swapper/211:48:052
3006699250,24cyclictest0-21swapper/212:30:332
3006099230,2cyclictest0-21swapper/012:39:310
30066992219,3cyclictest0-21swapper/212:39:432
30066992219,3cyclictest0-21swapper/211:24:232
30066992218,3cyclictest0-21swapper/210:35:132
30066992216,5cyclictest0-21swapper/210:42:332
3006299220,0cyclictest0-21swapper/107:40:121
3006099210,1cyclictest0-21swapper/010:57:530
30062992019,1cyclictest0-21swapper/109:37:131
3006299200,0cyclictest0-21swapper/109:15:371
30060992011,3cyclictest537-21snmpd10:24:040
30066991914,4cyclictest13986-21ssh10:01:332
3006699190,19cyclictest0-21swapper/211:42:562
3006999184,8cyclictest14212-21apt-get11:00:113
30066991818,0cyclictest0-21swapper/212:18:022
30066991817,1cyclictest30051-21cyclictest11:16:032
30066991815,3cyclictest0-21swapper/211:53:022
30066991814,4cyclictest0-21swapper/212:21:332
3006699181,17cyclictest0-21swapper/211:59:332
3006699181,17cyclictest0-21swapper/211:59:332
3006699180,17cyclictest0-21swapper/211:30:032
3006299187,2cyclictest23073-21diskmemload09:11:301
3006299181,17cyclictest19146-21ssh11:02:411
3006099185,5cyclictest225-21systemd-journal12:30:010
3006099181,12cyclictest0-21swapper/011:07:050
3006099180,7cyclictest0-21swapper/007:50:120
3006099180,15cyclictest0-21swapper/010:18:080
30066991715,2cyclictest0-21swapper/211:35:232
30066991715,2cyclictest0-21swapper/209:55:222
3006699170,17cyclictest0-21swapper/211:05:232
3006699170,17cyclictest0-21swapper/210:17:032
3006699170,17cyclictest0-21swapper/210:07:532
3006699170,17cyclictest0-21swapper/210:07:522
3006699170,0cyclictest0-21swapper/210:32:022
30062991714,2cyclictest0-21swapper/108:45:201
30060991716,1cyclictest0-21swapper/009:31:280
151792170,0sleep10-21swapper/111:57:581
151792170,0sleep10-21swapper/111:57:571
3006999166,5cyclictest225-21systemd-journal07:24:593
3006999162,6cyclictest0-21swapper/309:11:493
30069991610,6cyclictest30051-21cyclictest10:48:173
30066991616,0cyclictest0-21swapper/211:27:022
30066991615,1cyclictest55750irq/128-enp2s0-12:12:332
30066991615,1cyclictest0-21swapper/212:04:522
30066991615,1cyclictest0-21swapper/210:52:532
30066991613,3cyclictest0-21swapper/211:04:422
30066991613,2cyclictest309-21grep11:10:222
3006699160,2cyclictest0-21swapper/212:27:252
3006299161,13cyclictest23073-21diskmemload10:11:021
3006099164,7cyclictest22143-21idleruntime-cro11:04:590
30060991615,1cyclictest0-21swapper/009:15:280
3006099160,16cyclictest0-21swapper/012:01:080
3006999159,2cyclictest2139-21grep09:55:113
30069991513,1cyclictest0-21swapper/312:11:593
3006999150,0cyclictest0-21swapper/312:17:043
30066991514,1cyclictest55850irq/129-enp2s0-10:45:122
30066991514,1cyclictest0-21swapper/209:45:322
3006299152,12cyclictest23073-21diskmemload10:02:171
3006299151,14cyclictest9708-21sort09:20:261
3006099153,6cyclictest27943-21munin-run10:30:010
3006099153,6cyclictest19219-21munin-run12:20:000
3006099150,15cyclictest0-21swapper/010:03:150
155892150,0chrt0-21swapper/311:39:073
3006999149,3cyclictest27197-21wc09:50:183
3006999149,3cyclictest25321-21wc08:05:183
3006999141,4cyclictest225-21systemd-journal08:26:553
3006999141,4cyclictest0-21swapper/310:02:303
30069991411,2cyclictest14952-21tail10:40:263
30069991410,3cyclictest9808-21cut07:35:153
30069991410,3cyclictest30761-21grep08:15:263
30069991410,3cyclictest30761-21grep08:15:263
30069991410,3cyclictest17196-21sh12:37:323
30069991410,2cyclictest6651-21cut10:55:173
3006999140,1cyclictest0-21swapper/311:18:203
3006999140,13cyclictest10475-21grep08:40:263
30066991411,3cyclictest0-21swapper/209:30:522
30066991410,4cyclictest550-21lldpd09:29:522
3006299142,3cyclictest225-21systemd-journal09:05:001
30062991412,0cyclictest0-21swapper/109:49:581
30062991411,2cyclictest5920-21cron08:35:011
3006299141,12cyclictest23073-21diskmemload10:26:441
30062991410,4cyclictest0-21swapper/112:25:191
3006299140,0cyclictest0-21swapper/112:18:091
3006099149,3cyclictest1008-21tr09:35:140
3006099142,7cyclictest10058-21basename10:00:010
30060991410,3cyclictest4845-21wc08:30:190
30060991410,3cyclictest21133-21ssh12:20:210
30060991410,2cyclictest13343-21tr10:40:140
3006099140,2cyclictest0-21swapper/011:41:180
3006999139,3cyclictest3473-21grep11:50:273
3006999139,3cyclictest30232-21ls10:30:243
3006999139,3cyclictest2251-21grep11:30:253
3006999139,3cyclictest19210-21grep10:05:153
3006999139,3cyclictest19210-21grep10:05:143
3006999139,2cyclictest17557-21ssh11:40:143
3006999139,2cyclictest11906-21tail11:55:293
3006999139,2cyclictest11906-21tail11:55:293
3006999131,7cyclictest8448-21missed_timers07:30:173
30069991310,2cyclictest5291-21grep07:25:143
30069991310,2cyclictest29262-21head10:10:263
30069991310,2cyclictest2117-21grep09:35:213
30069991310,2cyclictest20192-21sed09:00:253
30069991310,2cyclictest1510-21wc07:15:213
3006699138,4cyclictest23669-21basename09:10:122
3006699132,1cyclictest0-21swapper/209:44:022
30066991310,2cyclictest9919-21ntpq08:40:202
30066991310,2cyclictest21013-21taskset09:05:132
30066991310,2cyclictest17967-21taskset10:23:462
3006699130,1cyclictest0-21swapper/209:50:122
3006699130,12cyclictest360-21grep09:15:172
3006699130,12cyclictest26984-21grep08:10:142
3006299139,3cyclictest3690-21taskset10:15:141
3006299138,4cyclictest1146-21cat07:15:181
30062991313,0cyclictest0-21swapper/110:30:131
30062991312,1cyclictest0-21swapper/111:40:151
3006299131,11cyclictest0-21swapper/109:44:081
30062991310,2cyclictest989-21cut11:30:151
30062991310,2cyclictest3492-21grep12:10:161
30062991310,2cyclictest28315-21grep08:10:261
30062991310,2cyclictest13142-21ssh10:20:211
30062991310,2cyclictest11990-21wc07:39:591
3006099139,3cyclictest24245-21latency_hist09:30:000
3006099139,3cyclictest16291-21grep07:45:270
3006099138,3cyclictest28447-21wc12:05:260
30060991310,2cyclictest6214-21wc10:35:250
30060991310,2cyclictest30961-21wc10:50:190
30060991310,2cyclictest29052-21ssh10:30:150
30060991310,2cyclictest26102-21sed11:25:260
30060991310,2cyclictest21134-21wc07:55:270
3006099130,13cyclictest0-21swapper/012:12:510
3006099130,13cyclictest0-21swapper/010:12:030
3006099130,12cyclictest20210-21tr10:05:210
3006099130,12cyclictest20210-21tr10:05:200
3006999129,2cyclictest5960-21tail12:30:253
3006999129,2cyclictest5622-21grep10:35:203
3006999129,2cyclictest28313-21sed08:10:263
3006999129,2cyclictest26144-21head11:25:263
3006999129,2cyclictest24252-21basename09:30:003
3006999129,2cyclictest21701-21tail09:05:173
3006999129,2cyclictest11293-21cat10:20:003
3006999128,2cyclictest16643-21cut11:20:183
30069991211,1cyclictest0-21swapper/311:47:483
30069991210,1cyclictest28344-21grep12:25:123
3006699129,2cyclictest7871-21grep08:35:262
3006699129,2cyclictest30783-21sed08:15:262
3006699129,2cyclictest30783-21sed08:15:252
3006699129,2cyclictest25009-21grep08:05:162
3006699129,2cyclictest2227-21head08:25:162
3006699128,1cyclictest19195-21cut07:55:012
3006299129,2cyclictest6865-21grep08:35:171
3006299129,2cyclictest6184-21cat10:35:251
3006299129,2cyclictest3868-21cat09:55:231
3006299129,2cyclictest31515-21grep11:10:151
3006299129,2cyclictest30610-21grep10:50:161
3006299129,2cyclictest28640-21grep12:05:271
3006299129,2cyclictest24771-21ssh12:03:551
3006299129,2cyclictest22720-21head08:00:161
3006299129,2cyclictest13560-21df10:40:151
3006299121,1cyclictest0-21swapper/109:29:361
3006299120,3cyclictest23657-21cron11:25:001
3006099129,2cyclictest7943-21grep07:30:140
3006099129,2cyclictest32681-21sed08:20:200
3006099129,2cyclictest31457-21sort11:10:150
3006099129,2cyclictest1930-21sed08:25:130
3006099129,2cyclictest18546-21grep09:45:160
3006099129,2cyclictest16729-21cut08:55:170
3006099129,2cyclictest10851-21sh11:55:200
3006099129,2cyclictest10851-21sh11:55:190
3006099129,2cyclictest10296-21cut09:40:190
3006099129,2cyclictest10080-21tr08:40:200
3006099128,3cyclictest15723-21ls08:55:000
3006099128,2cyclictest9027-21uname09:20:190
3006099123,8cyclictest21488-21aten2_r7power_c10:45:120
3006999118,2cyclictest942-21ssh11:10:333
3006999118,2cyclictest30924-21grep07:10:163
3006999118,2cyclictest30071-21ssh12:06:323
3006999118,2cyclictest29816-21ssh10:50:123
3006999118,2cyclictest27913-21ssh09:31:193
3006999118,2cyclictest24075-21grep08:04:593
3006999118,2cyclictest23432-21ssh11:05:153
3006999118,2cyclictest23070-21ssh10:26:153
3006999118,2cyclictest20289-21grep12:00:263
3006999118,2cyclictest20284-21tail07:55:183
3006999118,2cyclictest14149-21ssh09:42:353
3006999118,2cyclictest13561-21grep10:20:263
3006999118,2cyclictest12843-21cut07:40:163
3006999117,3cyclictest18499-21cut09:45:163
3006999112,4cyclictest23073-21diskmemload09:15:133
3006999110,10cyclictest4300-21chrt08:30:153
3006699119,1cyclictest1006-21grep07:15:172
3006699118,2cyclictest8835-21head09:20:182
3006699118,2cyclictest3583-21uname07:20:172
3006699118,2cyclictest26820-21tr12:05:152
3006699118,2cyclictest14638-21ls08:50:192
3006699116,4cyclictest0-21swapper/207:35:062
3006699114,3cyclictest225-21systemd-journal08:04:592
3006699113,6cyclictest12222-21apache_processe07:40:112
3006699112,8cyclictest23250-21kworker/u8:010:58:262
3006699111,1cyclictest6400-21mailstats07:25:212
3006699110,10cyclictest28157-21date10:30:012
3006299119,1cyclictest2260-21grep11:50:181
3006299119,1cyclictest19557-21df_inode10:05:161
3006299119,1cyclictest19557-21df_inode10:05:151
3006299118,2cyclictest8828-21fgrep11:15:201
3006299118,2cyclictest31377-21grep07:10:191
3006299118,2cyclictest25533-21basename11:45:121
3006299118,2cyclictest25357-21ssh11:05:511
3006299118,2cyclictest20295-21grep07:55:181
3006299118,2cyclictest18079-21cut07:50:171
3006299118,2cyclictest13117-21diskstats12:35:141
3006299118,2cyclictest11654-21ssh11:36:041
3006299118,2cyclictest11524-21ssh12:34:371
3006299118,2cyclictest10867-21rm10:57:471
3006299114,6cyclictest0-21swapper/112:20:131
3006299114,3cyclictest225-21systemd-journal08:25:011
3006299112,8cyclictest0-21swapper/110:45:121
3006099118,2cyclictest9371-21ssh12:33:000
3006099118,2cyclictest8999-21egrep11:35:140
3006099118,2cyclictest6939-21sed08:35:180
3006099118,2cyclictest4167-21ssh11:31:560
3006099118,2cyclictest28529-21chrt08:11:290
3006099118,2cyclictest26167-21grep11:45:160
3006099118,2cyclictest25470-21grep09:10:250
3006099118,2cyclictest2159-21ssh11:50:170
3006099118,2cyclictest19644-21ssh11:21:540
3006099118,2cyclictest12527-21cat07:40:140
3006099117,2cyclictest7639-21tr11:15:120
3006999108,1cyclictest31859-21expr08:20:133
3006999108,1cyclictest19460-21rm12:20:023
3006999108,1cyclictest17889-21chrt08:57:393
3006999108,1cyclictest13130-21taskset08:47:313
3006999107,2cyclictest17928-21wc07:50:183
3006999107,2cyclictest15225-21cut07:45:163
3006999106,2cyclictest7635-21wc08:35:253
3006699108,2cyclictest31284-21cat07:10:182
3006699108,1cyclictest7597-21grep07:30:132
3006699108,1cyclictest4361-21sed08:30:152
3006699108,1cyclictest215212sleep207:58:162
3006699108,1cyclictest18520-21cp09:00:042
3006699108,1cyclictest11489-21expr08:45:142
3006699107,2cyclictest17210-21head08:55:222
3006699107,2cyclictest1094-21taskset08:24:142
3006699106,3cyclictest16308-21grep07:45:262
3006299108,1cyclictest8629-21readlink08:40:001
3006299108,1cyclictest3093-21sed08:25:241
3006299108,1cyclictest29153-21tr08:15:141
3006299108,1cyclictest29153-21tr08:15:131
3006299108,1cyclictest2613-21taskset07:20:071
3006299108,1cyclictest24341-21chrt08:05:121
3006299108,1cyclictest24196-21sh11:25:121
3006299108,1cyclictest22507-21sort09:05:251
3006299108,1cyclictest16202-21tr08:55:151
3006299108,1cyclictest13618-21basename08:50:141
3006299107,2cyclictest7647-21tr07:30:131
3006299102,2cyclictest225-21systemd-journal07:28:301
3006299100,9cyclictest26388-21sh09:30:241
3006099108,1cyclictest30877-21entropy07:10:140
3006099107,2cyclictest2380-21munin-run07:20:000
3006099107,2cyclictest23161-21cut08:00:220
3006099107,2cyclictest20130-21sort09:00:260
3006099107,2cyclictest12186-21wc08:45:180
3006099106,2cyclictest24567-21grep08:05:130
3006099106,2cyclictest21103-21tr09:05:130
3006099101,4cyclictest0-21swapper/009:50:130
300699997,1cyclictest13681-21aten2_r7power_p08:50:143
300629996,2cyclictest15337-21grep07:45:171
300609997,1cyclictest29114-21cut08:15:140
300609997,1cyclictest29114-21cut08:15:130
300609997,1cyclictest11286-21sed07:35:250
300609996,2cyclictest6844-21chrt07:26:290
300609986,1cyclictest3034-21expr07:20:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional