You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-22 - 08:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot2s.osadl.org (updated Wed Apr 22, 2026 00:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287331997657,14cyclictest86-21hwrng22:46:570
287331997568,5cyclictest86-21hwrng00:08:520
287331997568,5cyclictest86-21hwrng00:08:520
287334997365,5cyclictest86-21hwrng23:09:293
287334997365,5cyclictest86-21hwrng23:09:293
287333997367,3cyclictest0-21swapper/200:23:132
287334997263,6cyclictest86-21hwrng20:10:173
287334997061,5cyclictest86-21hwrng21:29:083
287333996862,3cyclictest0-21swapper/223:38:092
287333996862,3cyclictest0-21swapper/223:38:092
287332996862,3cyclictest0-21swapper/100:23:131
287334996758,6cyclictest86-21hwrng00:17:043
287334996758,6cyclictest86-21hwrng00:17:043
287334996658,6cyclictest86-21hwrng22:18:173
287331996559,4cyclictest86-21hwrng22:44:540
287331996557,4cyclictest86-21hwrng20:45:060
287333996357,3cyclictest0-21swapper/223:44:182
287331996255,4cyclictest86-21hwrng23:43:160
287334996151,7cyclictest86-21hwrng21:59:513
287334996151,7cyclictest86-21hwrng21:59:513
287333996156,3cyclictest0-21swapper/222:13:102
287331996150,8cyclictest23-21rcuc/023:02:550
287331996149,9cyclictest23-21rcuc/000:11:540
287331996148,10cyclictest23-21rcuc/022:25:120
287331996047,8cyclictest359608-21ssh23:36:460
287331996047,8cyclictest359608-21ssh23:36:460
287334995951,5cyclictest86-21hwrng22:07:013
287334995950,6cyclictest86-21hwrng22:23:243
287334995950,5cyclictest86-21hwrng20:24:373
287331995949,7cyclictest23-21rcuc/000:16:330
287331995949,7cyclictest23-21rcuc/000:16:330
287331995949,5cyclictest326172-21ssh22:22:400
287331995948,8cyclictest23-21rcuc/023:23:350
287331995948,7cyclictest292527-21modprobe20:52:550
287331995850,4cyclictest23-21rcuc/022:07:310
287331995849,6cyclictest23-21rcuc/023:48:370
287331995849,5cyclictest23-21rcuc/021:59:310
287331995849,5cyclictest23-21rcuc/021:59:310
287331995847,8cyclictest23-21rcuc/021:35:470
287333995751,3cyclictest0-21swapper/222:39:472
287331995749,5cyclictest86-21hwrng23:53:310
287331995749,5cyclictest86-21hwrng23:15:380
287334995648,6cyclictest86-21hwrng20:17:273
287331995649,4cyclictest331164-21ssh22:33:560
287331995649,4cyclictest313314-21ssh21:54:100
287331995649,4cyclictest313314-21ssh21:54:100
287331995647,6cyclictest23-21rcuc/000:04:500
287331995546,6cyclictest23-21rcuc/022:11:320
287331995448,4cyclictest23-21rcuc/022:00:360
287331995448,3cyclictest23-21rcuc/022:51:550
287331995245,4cyclictest86-21hwrng22:55:090
287331995245,4cyclictest86-21hwrng22:55:090
287334995141,6cyclictest86-21hwrng23:57:373
287334995041,6cyclictest86-21hwrng23:10:303
287334994839,6cyclictest86-21hwrng21:48:353
287334994837,7cyclictest86-21hwrng19:12:563
287333994831,13cyclictest0-21swapper/220:40:032
287334994739,5cyclictest86-21hwrng00:37:333
287334994739,5cyclictest86-21hwrng00:37:333
287334994720,9cyclictest0-21swapper/323:45:023
287331994737,7cyclictest299702-21ssh21:24:040
287331994737,7cyclictest299702-21ssh21:24:040
287334994638,5cyclictest86-21hwrng22:58:133
287334994638,5cyclictest86-21hwrng22:58:133
287334994638,5cyclictest86-21hwrng21:25:023
287334994638,5cyclictest86-21hwrng21:25:023
287333994640,3cyclictest0-21swapper/200:26:172
28733499452,21cyclictest288824-21munin-run19:40:023
287334994437,5cyclictest86-21hwrng22:25:273
287334994437,5cyclictest86-21hwrng00:02:443
287331994438,4cyclictest86-21hwrng00:35:300
287331994438,4cyclictest86-21hwrng00:35:300
287334994329,10cyclictest330326-21kworker/u16:9+xprtiod22:40:183
287334994217,7cyclictest0-21swapper/321:10:203
287331994220,16cyclictest354417-21ssh23:25:080
28733199421,21cyclictest292798-21munin-run21:00:010
287334994119,17cyclictest351473-21kworker/u16:10+xprtiod23:25:123
28733499411,34cyclictest354167-21apt-get23:25:013
287333994119,16cyclictest0-21swapper/219:35:412
287332994135,2cyclictest0-21swapper/100:26:171
287331994134,5cyclictest86-21hwrng21:14:480
287334994032,5cyclictest86-21hwrng00:27:183
28733499401,32cyclictest318322-21cat22:05:023
28733499401,27cyclictest330326-21kworker/u16:9+rpciod22:40:053
287331994016,14cyclictest307539-21ssh21:41:110
287331994016,14cyclictest307539-21ssh21:41:110
287334993926,10cyclictest0-21swapper/300:23:543
287334993925,10cyclictest0-21swapper/322:50:013
287334993920,14cyclictest350286-21ssh23:16:053
287334993919,18cyclictest0-21swapper/321:35:013
287334993919,18cyclictest0-21swapper/321:35:013
287334993917,16cyclictest306880-21sendmail-msp21:40:053
287334993917,16cyclictest306880-21sendmail-msp21:40:053
28733499391,32cyclictest292294-21sshd20:48:563
28733499391,16cyclictest313668-21munin-run21:55:023
28733499391,16cyclictest313668-21munin-run21:55:023
287333993933,3cyclictest0-21swapper/220:50:132
28733199391,34cyclictest303129-21ssh21:31:300
28733199391,34cyclictest303129-21ssh21:31:300
28733199391,17cyclictest290824-21munin-run20:20:020
287334993828,6cyclictest86-21hwrng19:25:143
287334993826,9cyclictest0-21swapper/300:06:383
287334993826,9cyclictest0-21swapper/300:06:383
287334993825,10cyclictest0-21swapper/300:35:023
28733499382,13cyclictest345217-21ssh23:05:023
287334993820,13cyclictest320836-21ssh22:10:403
28733499381,26cyclictest288065-21kworker/u16:11+rpciod19:25:033
28733499381,24cyclictest292042-21kworker/u16:6+rpciod21:05:023
28733499381,16cyclictest375704-21sh00:13:013
287333993821,13cyclictest0-21swapper/221:00:022
287333993818,17cyclictest0-21swapper/222:18:202
287331993816,18cyclictest347740-21ssh23:10:160
287331993814,14cyclictest291318-21unix_chkpwd20:30:000
287334993724,10cyclictest0-21swapper/321:15:563
287334993718,15cyclictest356937-21ssh23:30:453
287334993718,15cyclictest356937-21ssh23:30:453
28733499371,31cyclictest1282-21snmpd22:51:053
28733499371,29cyclictest331734-21latency_hist22:35:023
28733499371,24cyclictest287789-21kworker/u16:3+rpciod19:20:023
28733499371,12cyclictest290050-21sshd-auth20:01:553
287333993731,3cyclictest0-21swapper/200:07:512
287333993731,3cyclictest0-21swapper/200:07:512
287331993728,4cyclictest86-21hwrng21:05:350
287331993728,4cyclictest86-21hwrng21:05:350
287331993724,10cyclictest311487-21latency_hist21:50:020
287331993715,17cyclictest347329-21ssh23:09:460
287331993715,17cyclictest347329-21ssh23:09:460
28733199371,31cyclictest379295-21ssh00:20:490
28733199371,15cyclictest289831-21sendmail-msp20:00:010
28733199371,13cyclictest292792-21apt-get21:00:150
287331993710,7cyclictest23-21rcuc/000:27:100
287334993619,13cyclictest364854-21ssh23:48:493
28733499361,30cyclictest293512-21latency_hist21:10:033
28733499361,30cyclictest293512-21latency_hist21:10:033
28733499361,17cyclictest360960-21munin-run23:40:013
28733499361,17cyclictest360960-21munin-run23:40:013
28733499361,17cyclictest291889-21latency_hist20:40:033
287333993630,3cyclictest0-21swapper/219:54:552
287333993623,10cyclictest0-21swapper/222:24:552
287333993620,13cyclictest0-21swapper/222:05:022
287333993619,13cyclictest0-21swapper/221:30:012
287331993628,6cyclictest23-21rcuc/021:15:530
28733199361,30cyclictest301079-21sshd-auth21:26:560
287334993525,8cyclictest0-21swapper/323:50:543
28733499352,23cyclictest306859-21munin-run21:40:023
28733499352,16cyclictest288574-21unix_chkpwd19:35:003
287333993520,11cyclictest0-21swapper/223:01:552
287333993520,11cyclictest0-21swapper/222:31:132
287332993525,7cyclictest0-21swapper/120:40:031
287331993516,14cyclictest358705-21ssh23:35:020
287331993516,14cyclictest358705-21ssh23:35:020
28733199351,30cyclictest367982-21ssh23:55:340
28733199351,30cyclictest332194-21modprobe22:35:550
28733199351,14cyclictest291961-21latency_hist20:40:030
28733499345,18cyclictest291316-21cron20:30:003
287334993422,8cyclictest0-21swapper/320:10:013
28733499341,13cyclictest292895-21cat21:00:033
287333993420,11cyclictest0-21swapper/221:45:552
287333993420,11cyclictest0-21swapper/221:32:562
287333993420,11cyclictest0-21swapper/221:32:562
287333993419,11cyclictest0-21swapper/221:42:372
287333993419,11cyclictest0-21swapper/221:42:372
287333993418,12cyclictest0-21swapper/220:15:022
287333993417,13cyclictest0-21swapper/222:56:382
287333993417,13cyclictest0-21swapper/222:56:382
287333993417,13cyclictest0-21swapper/200:00:002
287333993416,14cyclictest0-21swapper/223:10:032
287331993424,6cyclictest23-21rcuc/019:20:030
287331993413,17cyclictest384513-21ssh00:32:460
28733199341,30cyclictest323995-21ssh22:17:560
28733499333,19cyclictest289770-21systemd-userwor19:55:463
28733499332,17cyclictest291657-21latency_hist20:35:023
287334993318,12cyclictest289086-21latency_hist19:45:013
287333993322,8cyclictest0-21swapper/223:18:382
287333993321,8cyclictest0-21swapper/200:11:562
287333993320,10cyclictest0-21swapper/223:25:102
287333993319,11cyclictest0-21swapper/223:20:302
287333993319,11cyclictest0-21swapper/222:40:132
287333993319,10cyclictest0-21swapper/222:46:332
287333993319,10cyclictest0-21swapper/222:07:372
287333993319,10cyclictest0-21swapper/221:16:382
287333993318,12cyclictest0-21swapper/222:27:392
287333993318,11cyclictest0-21swapper/223:46:472
287333993318,11cyclictest0-21swapper/220:20:012
287333993318,11cyclictest0-21swapper/200:32:202
287333993317,11cyclictest0-21swapper/221:55:102
287333993317,11cyclictest0-21swapper/221:55:102
287333993314,15cyclictest0-21swapper/200:39:472
287333993314,15cyclictest0-21swapper/200:39:472
287332993317,12cyclictest0-21swapper/100:34:501
287331993317,11cyclictest291056-21kworker/u16:9+rpciod20:33:580
287331993316,13cyclictest291086-21munin-run20:25:020
287333993226,3cyclictest0-21swapper/221:35:492
287333993218,11cyclictest0-21swapper/223:30:432
287333993218,11cyclictest0-21swapper/223:30:432
287333993218,11cyclictest0-21swapper/221:10:342
287333993218,10cyclictest0-21swapper/221:52:362
287333993218,10cyclictest0-21swapper/221:52:362
287333993217,11cyclictest0-21swapper/220:00:022
287333993217,11cyclictest0-21swapper/200:01:562
287333993214,14cyclictest0-21swapper/221:23:102
287333993214,14cyclictest0-21swapper/221:23:102
287333993214,14cyclictest0-21swapper/221:05:022
287333993214,14cyclictest0-21swapper/219:25:032
287332993219,9cyclictest0-21swapper/123:10:571
28733199329,20cyclictest0-21swapper/020:04:560
28733499312,12cyclictest292542-21cron20:55:003
28733499311,24cyclictest289309-21sshd-auth19:48:343
287333993119,9cyclictest0-21swapper/223:05:302
287333993119,8cyclictest0-21swapper/223:51:142
287333993118,9cyclictest0-21swapper/222:50:522
287333993118,10cyclictest0-21swapper/220:35:022
287333993115,13cyclictest0-21swapper/220:10:022
287333993115,12cyclictest0-21swapper/221:10:012
287333993115,12cyclictest0-21swapper/221:10:012
287333993115,12cyclictest0-21swapper/219:45:562
287333993115,12cyclictest0-21swapper/200:16:432
287333993115,12cyclictest0-21swapper/200:16:432
287332993120,8cyclictest0-21swapper/121:20:551
287332993120,8cyclictest0-21swapper/121:20:551
287332993113,14cyclictest0-21swapper/119:28:331
287331993117,9cyclictest23-21rcuc/019:43:590
287331993117,10cyclictest23-21rcuc/019:53:000
287331993115,9cyclictest23-21rcuc/019:28:130
287331993114,12cyclictest290567-21latency_hist20:15:030
28733199311,27cyclictest23-21rcuc/019:10:020
28733199311,21cyclictest290288-21systemd-userwor20:05:480
287334993019,5cyclictest11150irq/205-8000000.ethernet-rx020:44:103
287334993011,9cyclictest0-21swapper/319:55:033
287333993014,12cyclictest0-21swapper/219:20:022
287332993025,2cyclictest0-21swapper/119:54:551
287332993014,11cyclictest0-21swapper/122:56:411
287332993014,11cyclictest0-21swapper/122:56:411
287332993013,13cyclictest0-21swapper/123:17:051
287331993019,8cyclictest23-21rcuc/019:12:490
287331993016,10cyclictest23-21rcuc/019:34:070
28733199301,21cyclictest288781-21systemd-userwor19:35:410
287333992924,3cyclictest0-21swapper/220:02:062
287333992918,9cyclictest0-21swapper/223:10:372
287333992917,9cyclictest0-21swapper/220:22:022
287333992914,12cyclictest0-21swapper/220:50:022
287333992911,14cyclictest0-21swapper/219:35:022
287333992910,15cyclictest0-21swapper/219:10:022
28733299298,16cyclictest0-21swapper/123:40:151
28733299298,14cyclictest0-21swapper/121:00:591
287332992918,7cyclictest0-21swapper/122:24:551
287332992916,9cyclictest0-21swapper/122:10:371
287332992916,10cyclictest0-21swapper/122:02:291
287332992915,10cyclictest0-21swapper/123:01:551
287332992914,9cyclictest0-21swapper/121:57:111
287332992914,9cyclictest0-21swapper/121:57:111
287334992820,5cyclictest287323-21munin-run19:10:013
287333992813,11cyclictest0-21swapper/220:40:042
287333992812,12cyclictest0-21swapper/220:30:022
28733299288,15cyclictest0-21swapper/120:20:511
28733299285,16cyclictest0-21swapper/120:55:581
287332992816,9cyclictest0-21swapper/121:17:511
287332992814,10cyclictest0-21swapper/121:30:161
287332992814,10cyclictest0-21swapper/121:30:161
287332992811,12cyclictest0-21swapper/122:33:311
287333992711,12cyclictest0-21swapper/219:45:012
287333992711,12cyclictest0-21swapper/219:30:032
28733299277,15cyclictest0-21swapper/100:16:401
28733299277,15cyclictest0-21swapper/100:16:401
28733299276,15cyclictest0-21swapper/121:10:551
287332992716,8cyclictest0-21swapper/123:56:311
287332992714,8cyclictest0-21swapper/119:35:411
287332992713,9cyclictest0-21swapper/119:25:011
287332992713,8cyclictest0-21swapper/122:18:201
287332992713,11cyclictest0-21swapper/100:00:051
287332992713,10cyclictest0-21swapper/100:38:231
287332992713,10cyclictest0-21swapper/100:38:231
287332992712,11cyclictest0-21swapper/121:50:501
287332992712,11cyclictest0-21swapper/121:50:501
28733199271,23cyclictest291055-21kworker/u16:8+rpciod20:40:040
28733199271,22cyclictest289304-21modprobe19:47:560
28733199271,21cyclictest288050-21systemd-userwor19:20:380
287333992615,9cyclictest0-21swapper/219:11:422
28733299269,13cyclictest0-21swapper/123:06:261
28733299269,13cyclictest0-21swapper/123:06:261
28733299269,13cyclictest0-21swapper/120:50:571
28733299267,14cyclictest0-21swapper/122:40:091
28733299267,14cyclictest0-21swapper/122:06:131
28733299267,14cyclictest0-21swapper/121:36:071
28733299267,14cyclictest0-21swapper/120:15:501
28733299265,16cyclictest0-21swapper/119:41:461
28733299264,16cyclictest0-21swapper/120:10:491
287332992618,4cyclictest0-21swapper/123:31:181
287332992618,4cyclictest0-21swapper/123:31:181
287332992615,7cyclictest0-21swapper/123:25:101
287332992612,11cyclictest0-21swapper/120:45:561
287332992611,11cyclictest0-21swapper/123:36:291
287332992611,11cyclictest0-21swapper/123:36:291
287332992611,11cyclictest0-21swapper/122:47:191
28733299257,13cyclictest0-21swapper/120:30:531
28733299256,15cyclictest0-21swapper/119:45:441
28733299256,14cyclictest0-21swapper/123:51:351
28733299255,15cyclictest0-21swapper/123:45:561
287332992518,4cyclictest0-21swapper/121:40:271
287332992518,4cyclictest0-21swapper/121:40:271
287332992514,7cyclictest0-21swapper/122:50:321
287332992514,7cyclictest0-21swapper/121:30:011
287332992511,9cyclictest0-21swapper/122:35:121
287332992511,10cyclictest0-21swapper/123:22:121
28733299248,12cyclictest0-21swapper/100:05:541
28733299248,12cyclictest0-21swapper/100:05:541
28733299245,14cyclictest0-21swapper/122:26:171
28733299244,15cyclictest0-21swapper/119:20:011
28733299243,15cyclictest0-21swapper/119:35:011
287332992418,3cyclictest0-21swapper/100:13:301
28733299237,11cyclictest0-21swapper/120:30:021
28733299236,13cyclictest0-21swapper/120:10:011
28733299234,14cyclictest0-21swapper/121:45:191
28733299226,14cyclictest0-21swapper/119:55:461
28733299225,13cyclictest0-21swapper/121:10:021
28733299225,13cyclictest0-21swapper/121:10:021
28733299225,13cyclictest0-21swapper/120:00:471
28733299225,12cyclictest0-21swapper/119:10:021
28733299215,14cyclictest0-21swapper/120:41:531
28733299215,12cyclictest0-21swapper/119:11:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional