You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 05:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 13, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2278329280,8sleep20-21swapper/219:05:342
2294029166,9sleep30-21swapper/319:07:363
2289029149,38sleep00-21swapper/019:06:580
2289628467,13sleep10-21swapper/119:07:021
2321499281,2cyclictest28209-21needreboot00:10:222
2320599270,26cyclictest0-21swapper/021:46:450
52902250,0sleep20-21swapper/223:37:392
2321499250,22cyclictest0-21swapper/223:13:302
2320599250,23cyclictest0-21swapper/022:31:590
2320599250,1cyclictest0-21swapper/021:50:260
2321499240,24cyclictest0-21swapper/221:24:052
2321499240,24cyclictest0-21swapper/200:01:262
2320599240,2cyclictest0-21swapper/022:45:480
2320599230,23cyclictest0-21swapper/000:13:440
2320599230,23cyclictest0-21swapper/000:02:390
2320599230,1cyclictest0-21swapper/022:25:570
2320599230,1cyclictest0-21swapper/022:07:030
2321699222,19cyclictest537-21snmpd23:51:083
2321699220,19cyclictest0-21swapper/321:05:153
23214992221,1cyclictest0-21swapper/221:41:242
2321499220,22cyclictest0-21swapper/222:53:312
2321499220,22cyclictest0-21swapper/222:31:592
2321499220,21cyclictest0-21swapper/223:45:282
2321499220,1cyclictest0-21swapper/221:12:532
2320599220,2cyclictest0-21swapper/023:21:210
2320599220,21cyclictest0-21swapper/022:53:300
2321499211,1cyclictest0-21swapper/223:29:262
2321499210,1cyclictest0-21swapper/200:21:012
2321499210,1cyclictest0-21swapper/200:19:422
2320999211,18cyclictest0-21swapper/123:27:321
2321499200,20cyclictest0-21swapper/222:37:392
2321499200,19cyclictest0-21swapper/200:35:432
2320999200,20cyclictest0-21swapper/121:36:341
2320599200,1cyclictest0-21swapper/023:54:220
2320599200,19cyclictest0-21swapper/000:23:190
2320599200,17cyclictest0-21swapper/000:31:270
55282190,0sleep30-21swapper/322:20:173
2321499191,17cyclictest0-21swapper/223:40:392
2321499190,19cyclictest0-21swapper/220:11:002
2321499190,14cyclictest0-21swapper/200:05:102
2320999190,2cyclictest0-21swapper/123:50:201
2320999190,19cyclictest0-21swapper/121:27:051
2320599190,2cyclictest0-21swapper/023:29:260
2320599190,19cyclictest0-21swapper/021:39:000
2320599190,18cyclictest0-21swapper/021:41:230
71762180,0sleep10-21swapper/122:20:481
2321699181,15cyclictest0-21swapper/323:15:143
2321699181,13cyclictest15609-21awk23:25:003
2321499180,18cyclictest0-21swapper/222:45:302
2321499180,18cyclictest0-21swapper/221:19:182
2321499180,17cyclictest0-21swapper/222:25:532
23209991814,2cyclictest0-21swapper/122:41:131
2320599181,3cyclictest0-21swapper/023:00:130
2320599180,18cyclictest0-21swapper/023:09:280
2320599180,18cyclictest0-21swapper/021:24:310
2320599180,17cyclictest0-21swapper/023:35:410
2320599180,14cyclictest0-21swapper/022:40:460
2321699170,16cyclictest0-21swapper/323:34:343
2321499172,2cyclictest0-21swapper/221:35:132
2321499170,1cyclictest0-21swapper/200:31:262
2320999175,1cyclictest141rcu_preempt23:42:051
2320599170,13cyclictest0-21swapper/021:27:540
2320599170,0cyclictest0-21swapper/023:14:100
2321699163,2cyclictest0-21swapper/300:30:403
2321699160,15cyclictest0-21swapper/320:05:143
2321499166,7cyclictest13873-21munin-node21:49:552
23214991615,1cyclictest0-21swapper/221:31:222
2321499161,2cyclictest0-21swapper/223:56:452
2320999166,2cyclictest0-21swapper/100:00:011
23205991615,1cyclictest0-21swapper/021:11:400
2320599160,1cyclictest0-21swapper/022:37:390
2320599160,12cyclictest0-21swapper/022:20:150
2321699152,12cyclictest0-21swapper/322:15:313
23216991512,2cyclictest22895-21cron23:09:593
23216991511,2cyclictest27439-21cat21:55:013
23214991514,1cyclictest0-21swapper/223:50:262
23214991510,4cyclictest19475-21grep21:50:132
2321499150,2cyclictest0-21swapper/222:20:152
2321499150,1cyclictest0-21swapper/222:42:502
2321499150,14cyclictest0-21swapper/221:27:542
2321499150,0cyclictest0-21swapper/219:45:122
2320999158,6cyclictest38150irq/131-snd_hda22:59:171
2320999153,8cyclictest225-21systemd-journal00:10:151
23209991514,1cyclictest0-21swapper/122:32:501
2320599157,7cyclictest1667-21munin-run20:40:010
2320599157,7cyclictest1667-21munin-run20:40:000
23205991513,1cyclictest22868-21sshd23:48:090
2320599150,11cyclictest0-21swapper/022:00:040
2321699142,8cyclictest0-21swapper/300:36:303
2321699141,2cyclictest225-21systemd-journal00:09:263
23216991411,2cyclictest30304-21head20:30:183
2321699141,12cyclictest17393-21kworker/u8:223:37:083
23214991413,1cyclictest0-21swapper/221:57:232
2321499140,13cyclictest0-21swapper/222:00:582
2320999148,5cyclictest3833-21apache223:12:451
23209991411,2cyclictest24293-21grep22:50:261
2320999140,9cyclictest0-21swapper/121:45:041
2320999140,1cyclictest0-21swapper/100:05:421
2320599148,1cyclictest31805-21apt-get23:15:140
23205991412,1cyclictest16218-21diskmemload00:05:070
23205991411,2cyclictest28422-21ssh21:16:280
140192143,2sleep114027-21apt-get22:45:131
2321699139,3cyclictest21651-21grep22:30:163
2321699131,8cyclictest0-21swapper/321:35:143
23216991310,2cyclictest18009-21chrt22:27:473
2321699130,12cyclictest508-21basename21:20:003
2321699130,11cyclictest23159-21grep20:15:183
2321499136,7cyclictest0-21swapper/223:04:042
2321499130,13cyclictest0-21swapper/223:33:422
2321499130,13cyclictest0-21swapper/222:09:032
2321499130,12cyclictest0-21swapper/221:05:222
2320999139,2cyclictest4295-21wc20:45:011
2320999137,4cyclictest23196-21cyclictest23:47:551
23209991313,0cyclictest0-21swapper/119:40:161
23209991310,2cyclictest17653-21wc21:10:191
23209991310,2cyclictest17207-21cron20:05:001
2320999130,8cyclictest0-21swapper/121:55:261
2320999130,12cyclictest0-21swapper/122:00:431
23205991312,1cyclictest0-21swapper/021:33:540
23205991310,2cyclictest25564-21grep19:15:010
23205991310,2cyclictest23161-21sh22:11:040
2320599130,3cyclictest0-21swapper/000:27:550
2320599130,0cyclictest0-21swapper/023:58:170
2321699129,2cyclictest31236-21ssh23:14:593
2321699129,2cyclictest2762-21cat19:35:003
2321699129,2cyclictest26148-21uname20:20:233
2321699129,2cyclictest17245-21cat20:04:593
2321699129,2cyclictest16869-21grep23:25:183
2321699129,2cyclictest11960-21tr21:00:113
2321699129,2cyclictest11496-21tr00:20:143
2321699128,3cyclictest3981-21cut22:00:133
2321699128,3cyclictest26707-21ssh00:10:113
2321699128,3cyclictest14803-21head22:45:163
2321699128,2cyclictest794-21wc20:35:233
2321699128,2cyclictest794-21wc20:35:223
2321699128,2cyclictest13723-21sed22:45:013
2321699123,4cyclictest225-21systemd-journal21:35:003
2321699123,4cyclictest0-21swapper/321:10:243
2321499129,2cyclictest4123-21perf20:45:012
2321499129,2cyclictest20861-21cat22:10:152
2321499129,2cyclictest12961-21head21:00:192
2321499124,2cyclictest225-21systemd-journal19:11:092
23214991210,1cyclictest31965-21grep23:15:152
23214991210,1cyclictest17006-21grep23:05:262
2321499120,4cyclictest0-21swapper/219:35:122
2321499120,1cyclictest0-21swapper/220:35:122
2321499120,1cyclictest0-21swapper/220:35:112
2321499120,1cyclictest0-21swapper/200:29:262
2321499120,12cyclictest0-21swapper/222:15:052
2320999129,2cyclictest9765-21grep23:20:261
2320999129,2cyclictest27523-21grep19:15:261
2320999129,2cyclictest24325-21ssh22:11:571
2320999129,2cyclictest2074-21grep21:20:171
2320999129,1cyclictest16218-21diskmemload22:25:541
2320999128,3cyclictest30175-21grep22:15:201
2320999128,3cyclictest28279-21wc20:25:191
2320999122,9cyclictest19235-21taskset22:09:311
23209991212,0cyclictest0-21swapper/121:44:121
23209991210,1cyclictest11863-21grep21:00:111
2320999120,1cyclictest141rcu_preempt20:55:291
2320999120,11cyclictest30195-21expr22:35:151
2320999120,11cyclictest1305-21tail23:35:191
2320599129,2cyclictest32563-21ls22:55:270
2320599129,2cyclictest32376-21ssh21:57:310
2320599129,2cyclictest2674-21ssh00:15:060
2320599129,2cyclictest2056-21ssh22:18:220
2320599123,2cyclictest10134-21/usr/sbin/munin23:40:200
2320599121,5cyclictest225-21systemd-journal20:25:000
2320599120,12cyclictest0-21swapper/023:31:360
2321699119,1cyclictest117452sleep300:00:283
2321699118,2cyclictest9648-21sh21:25:043
2321699118,2cyclictest5180-21chrt21:40:473
2321699118,2cyclictest5101-21ssh00:15:373
2321699118,2cyclictest32539-21chrt19:28:033
2321699118,2cyclictest31633-21ssh21:56:553
2321699118,2cyclictest3161-21grep21:20:253
2321699118,2cyclictest30516-21ssh22:55:043
2321699118,2cyclictest28270-21cut19:20:153
2321699118,2cyclictest2802-21wc23:55:193
2321699118,2cyclictest26316-21df19:15:153
2321699118,2cyclictest25125-21sshd00:28:133
2321699118,2cyclictest24201-21sed19:10:193
2321699118,2cyclictest22474-21tr22:50:143
2321699118,2cyclictest20198-21ssh23:46:043
2321699118,2cyclictest13441-21ssh23:04:083
2321699118,2cyclictest13069-21sed21:45:273
2321699118,2cyclictest12510-21cut19:55:163
2321699118,2cyclictest11767-21ssh22:05:013
2321699118,2cyclictest11203-21ssh23:40:323
2321699118,2cyclictest10239-21cut19:50:143
2321699117,3cyclictest20651-21grep22:10:153
2321499119,2cyclictest16598-21sed20:00:252
2321499119,1cyclictest8945-21taskset20:53:332
2321499119,1cyclictest28176-21chrt19:20:022
2321499119,1cyclictest1400-21ssh22:56:182
2321499119,1cyclictest11149-21wc19:50:192
2321499118,2cyclictest674-21grep19:30:132
2321499118,2cyclictest26081-21tr19:15:142
2321499118,2cyclictest18677-21ls20:05:222
2321499118,2cyclictest10073-21ssh23:20:392
2321499112,6cyclictest0-21swapper/219:25:122
2321499111,1cyclictest25910-21cat20:20:202
2321499110,1cyclictest0-21swapper/220:56:022
2321499110,1cyclictest0-21swapper/220:15:032
2320999119,1cyclictest4501-21ssh00:15:261
2320999119,1cyclictest28961-21grep19:20:181
2320999118,2cyclictest8671-21sed20:50:261
2320999118,2cyclictest7952-21sort23:00:201
2320999118,2cyclictest6445-21ssh23:57:321
2320999118,2cyclictest3895-21ssh00:35:121
2320999118,2cyclictest32277-21cat20:35:161
2320999118,2cyclictest32277-21cat20:35:151
2320999118,2cyclictest26945-21cron20:25:011
2320999118,2cyclictest25185-21ssh21:53:231
2320999118,2cyclictest25046-21expr23:30:151
2320999118,2cyclictest22591-21ssh00:26:151
2320999118,2cyclictest2222-21ssh23:16:071
2320999118,2cyclictest20328-21tr20:10:161
2320999118,2cyclictest19612-21grep21:30:231
2320999118,2cyclictest19590-21apt20:09:591
2320999118,2cyclictest16899-21cat23:05:271
2320999118,2cyclictest14627-21sleep19:59:591
2320999118,2cyclictest11204-21ssh00:20:121
2320999117,3cyclictest28055-21grep00:30:141
2320999113,1cyclictest16218-21diskmemload21:18:081
2320999111,2cyclictest5196-21latency_hist19:40:001
2320999111,2cyclictest225-21systemd-journal20:20:001
2320599119,2cyclictest9695-21taskset19:48:050
2320599119,1cyclictest7141-21sshd19:40:450
2320599119,1cyclictest6958-21apache_accesses20:50:110
2320599119,1cyclictest22403-21taskset20:15:080
2320599119,1cyclictest14929-21basename21:05:180
2320599119,1cyclictest12697-21sort19:55:150
2320599118,2cyclictest5078-21cron19:40:000
2320599118,2cyclictest4758-21expr20:45:160
2320599118,2cyclictest3814-21ssh00:35:100
2320599118,2cyclictest31858-21cat20:35:000
2320599118,2cyclictest21126-21cat20:10:220
2320599118,2cyclictest17439-21chrt20:05:020
2320599113,7cyclictest0-21swapper/020:25:130
2320599110,1cyclictest0-21swapper/019:25:130
2320599110,10cyclictest9714-21expr20:55:170
2321699108,1cyclictest9979-21tune2fs20:55:193
2321699108,1cyclictest8169-21expr19:45:143
2321699108,1cyclictest7212-21chrt19:41:373
2321699108,1cyclictest290202sleep320:26:093
2321699107,2cyclictest7877-21taskset20:50:193
2321699107,2cyclictest5110-21head20:45:183
2321699107,2cyclictest3009-21cut19:35:143
2321699107,2cyclictest2426-21df_abs20:40:163
2321699101,5cyclictest225-21systemd-journal22:35:003
2321699100,9cyclictest20269-21cat20:10:163
2321499109,1cyclictest5973-21fschecks_count19:40:162
2321499108,1cyclictest30679-21chrt20:30:212
2321499108,1cyclictest27862-21tune2fs20:25:162
2321499108,1cyclictest12928-21expr19:55:172
2321499107,2cyclictest5919-21wc20:45:272
2320999108,1cyclictest23582-21wget19:10:161
2320999108,1cyclictest11280-21ntpq19:50:221
2320999107,2cyclictest5278-21cat20:45:201
2320999107,2cyclictest31924-21mailstats19:25:221
2320999107,2cyclictest30352-21tail20:30:171
2320999107,2cyclictest14680-21sed21:05:171
2320999102,7cyclictest2666-21awk19:34:591
2320999101,2cyclictest225-21systemd-journal19:50:001
2320599109,1cyclictest11694-21taskset21:00:080
2320599108,2cyclictest884-21cat19:30:140
2320599108,1cyclictest28659-21cat19:20:150
2320599108,1cyclictest26074-21grep19:15:150
2320599108,1cyclictest15024-21cat20:00:010
2320599107,2cyclictest11157-21sort19:50:210
232059998,1cyclictest3070-21ntp_kernel_pll_20:40:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional