You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 14:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sun Aug 30, 2026 00:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3864125991091,105cyclictest0-21swapper/000:10:020
3864139998411,43cyclictest275-21dbus-daemon20:10:013
3864125998341,39cyclictest0-21swapper/019:15:020
3864125998276,3cyclictest0-21swapper/023:25:020
3864139997921,30cyclictest3864972-21sed19:25:013
3864135997827,4cyclictest131rcu_preempt19:40:032
386413999762,41cyclictest3892774-21ssh21:58:313
386413999732,35cyclictest3962268-21ssh00:26:383
3864135997335,33cyclictest3467602-21systemd-journal00:20:002
386413599722,6cyclictest3933254-21munin-run23:25:012
386413999712,38cyclictest3935536-21ssh23:29:513
386413999711,35cyclictest3941403-21ssh23:42:003
386413999711,35cyclictest3941403-21ssh23:42:003
386413999701,31cyclictest3886443-21ssh21:44:593
3864135997038,28cyclictest3467602-21systemd-journal22:40:022
3864135997035,31cyclictest3964432-21sshd00:31:122
3864135996925,40cyclictest30-21ksoftirqd/223:51:092
3864135996925,40cyclictest30-21ksoftirqd/223:51:092
3864125996937,19cyclictest12-21ksoftirqd/021:18:380
3864135996852,13cyclictest3946845-21kworker/u8:400:00:032
3864139996732,30cyclictest3467602-21systemd-journal19:35:013
386413999671,62cyclictest48-21kcompactd022:45:313
386413999671,33cyclictest3914159-21ssh22:44:183
386413999671,29cyclictest3918419-21ssh22:53:283
386412599677,7cyclictest121ksoftirqd/023:11:010
386413999662,32cyclictest3903203-21rm22:20:313
386413999662,29cyclictest3908719-21ssh22:32:303
386413999662,29cyclictest3908719-21ssh22:32:303
386413999661,60cyclictest3899749-21ssh22:13:223
3864139996611,27cyclictest3936630-1kworker/u9:223:39:423
386413599663,19cyclictest3866464-21grep19:55:022
386412599661,6cyclictest0-21swapper/000:29:390
386413999651,60cyclictest3887809-21ssh21:47:453
386413999642,31cyclictest3890912-21ssh21:54:303
386413999641,59cyclictest3896412-21ssh22:05:503
386413999641,29cyclictest3958498-21ssh00:18:453
3864135996428,32cyclictest3877054-21ssh21:24:352
3864135996428,32cyclictest3877054-21ssh21:24:352
3864125996428,7cyclictest12-21ksoftirqd/021:29:060
3864139996323,36cyclictest3868482-21latency_hist20:35:013
386413999631,58cyclictest3955117-21ssh00:11:133
386413999631,58cyclictest3928557-21ssh23:14:593
386413999631,34cyclictest3919736-21ssh22:55:553
386413599633,16cyclictest3869361-21latency_hist20:50:022
3864135996331,28cyclictest3954401-21latency_hist00:10:032
3864135996322,36cyclictest30-21ksoftirqd/222:28:302
386413099633,56cyclictest3930985-21ssh23:20:021
3864125996329,31cyclictest12-21ksoftirqd/022:20:430
386413999621,58cyclictest3883792-21ssh21:39:003
386413999621,57cyclictest3950254-21ssh00:00:433
386413999621,57cyclictest3947844-21ssh23:55:333
386413999621,57cyclictest3922025-21ssh23:00:443
386413999621,57cyclictest3894942-21ssh22:02:513
386413999621,57cyclictest3876418-21ssh21:23:033
386413999621,57cyclictest3876418-21ssh21:23:033
3864139996214,20cyclictest3870305-21diskmemload23:47:443
3864139996211,24cyclictest3870305-21diskmemload21:19:423
386413599623,17cyclictest3930938-21unin-run23:20:012
3864135996217,41cyclictest30-21ksoftirqd/223:25:532
3864139996117,39cyclictest275-21dbus-daemon19:30:013
386413999611,56cyclictest3966479-21ssh00:35:383
386413999611,56cyclictest3945842-21ssh23:51:243
386413999611,56cyclictest3945842-21ssh23:51:243
386413999611,56cyclictest3931685-21ssh23:21:103
386413999611,56cyclictest3900921-21ssh22:15:403
386413999611,56cyclictest3871245-21ssh21:11:453
386413599613,18cyclictest3912112-21latency_hist22:40:012
3864135996113,44cyclictest30-21ksoftirqd/200:36:032
3864125996123,9cyclictest12-21ksoftirqd/022:57:330
386413999602,54cyclictest3467602-21systemd-journal23:06:193
386413999601,55cyclictest3963985-21ssh00:30:063
386413999601,55cyclictest3960565-21ssh00:23:093
386413999601,55cyclictest3954129-21ssh00:09:273
386413999601,55cyclictest3880173-21ssh21:30:533
386413999601,55cyclictest3877763-21ssh21:25:353
386413999601,54cyclictest48-21kcompactd019:41:343
386413599601,18cyclictest3467602-21systemd-journal00:05:002
386413599601,17cyclictest3938686-21ssh23:36:152
386413999591,55cyclictest3928862-21ssh23:15:183
386413999591,55cyclictest3905635-21ssh22:25:463
386413999591,54cyclictest3936233-21ssh23:30:503
386413999591,54cyclictest3936233-21ssh23:30:503
386413999591,54cyclictest3910586-21ssh22:36:353
3864135995930,25cyclictest3899252-21sh22:12:072
386413599592,15cyclictest3918964-21ssh22:54:372
3864135995919,34cyclictest275-21dbus-daemon21:55:012
3864125995937,18cyclictest0-21swapper/023:06:310
3864125995927,28cyclictest12-21ksoftirqd/023:32:060
3864125995927,28cyclictest12-21ksoftirqd/023:32:060
3864125995915,12cyclictest3959073-21sed00:20:010
3864125995911,12cyclictest12-21ksoftirqd/019:40:020
3864135995830,24cyclictest3937786-21sh23:34:422
3864135995830,24cyclictest3937786-21sh23:34:422
3864135995829,24cyclictest3907672-21sh22:30:092
3864135995829,24cyclictest3907672-21sh22:30:092
3864135995825,28cyclictest30-21ksoftirqd/200:23:072
386413599582,15cyclictest3868780-21munin-run20:40:012
3864135995819,35cyclictest30-21ksoftirqd/223:42:402
3864135995819,35cyclictest30-21ksoftirqd/223:42:402
3864135995819,35cyclictest30-21ksoftirqd/220:00:022
3864135995819,35cyclictest30-21ksoftirqd/220:00:022
3864130995812,33cyclictest0-21swapper/123:00:021
3864125995827,15cyclictest3870318-21ssh21:10:010
3864125995822,32cyclictest12-21ksoftirqd/022:27:430
386413599573,16cyclictest3963120-21ssh00:28:352
386413599573,14cyclictest3870043-21modprobe21:05:002
386413599572,8cyclictest3898129-21cron22:10:002
386413599572,15cyclictest3871189-21ssh21:11:382
386413599572,14cyclictest3916700-21ssh22:49:432
3864135995719,34cyclictest3872894-21sh21:15:102
386413599571,13cyclictest3880304-21ssh21:31:082
386412599574,4cyclictest131rcu_preempt00:23:000
3864125995740,12cyclictest3866643-21cron20:00:000
3864125995740,12cyclictest3866643-21cron20:00:000
3864125995729,13cyclictest3946681-21ssh23:53:260
3864125995729,13cyclictest3946681-21ssh23:53:250
386412599572,51cyclictest3895141-21ssh22:03:250
3864125995721,32cyclictest12-21ksoftirqd/023:39:210
3864125995712,21cyclictest0-21swapper/023:04:460
386413599562,8cyclictest3926189-21cron23:10:002
386413599562,15cyclictest3920363-21ssh22:57:242
386413599562,14cyclictest3901791-21ssh22:17:452
386413599562,13cyclictest3904737-21ssh22:24:062
3864135995620,32cyclictest3887167-21rm21:46:002
3864135995616,35cyclictest0-21swapper/223:12:442
3864135995612,40cyclictest30-21ksoftirqd/221:44:162
386413599561,14cyclictest3895516-21sh22:04:122
386413599561,12cyclictest1-21systemd21:30:002
3864125995637,4cyclictest16850irq/134-800000000:00:020
3864125995628,24cyclictest3898042-21ssh22:09:500
3864125995619,33cyclictest151rcuc/022:14:520
386412599561,8cyclictest0-21swapper/023:20:150
386412599561,38cyclictest3892561-21ssh21:57:490
386413999551,48cyclictest48-21kcompactd019:50:593
386413599552,17cyclictest3865367-21munin-run19:34:592
386413599552,13cyclictest3865115-21munin-run19:30:002
386413599551,16cyclictest3922997-21ssh23:03:062
386413599551,14cyclictest3955666-21ssh00:12:332
386413599551,13cyclictest3882154-21ssh21:35:052
386413599551,11cyclictest364-21snmpd21:59:072
386412599556,9cyclictest12-21ksoftirqd/023:42:540
386412599556,9cyclictest12-21ksoftirqd/023:42:540
3864125995545,5cyclictest16850irq/134-800000000:40:000
386413999541,49cyclictest48-21kcompactd021:02:003
386413599543,7cyclictest3868371-21grep20:30:022
386413599542,7cyclictest3467602-21systemd-journal19:15:002
386413599542,13cyclictest3866242-21latency_hist19:50:022
386413599542,13cyclictest3866242-21latency_hist19:50:022
3864125995426,10cyclictest3884390-21latency_hist21:40:020
386412599541,37cyclictest3914965-21ssh22:45:390
386413999531,48cyclictest364-21snmpd20:56:203
386413999531,47cyclictest364-21snmpd20:41:163
3864135995319,29cyclictest309-21in:imuxsock21:10:012
3864125995326,11cyclictest3876136-21ssh21:22:170
3864125995326,11cyclictest3876136-21ssh21:22:170
3864125995325,12cyclictest3928868-21ssh23:15:190
3864125995324,13cyclictest3912465-21ssh22:40:210
3864125995317,11cyclictest12-21ksoftirqd/022:35:020
3864125995317,11cyclictest12-21ksoftirqd/022:35:020
386413999521,47cyclictest3868232-21munin-run20:30:013
386413999521,46cyclictest48-21kcompactd020:53:413
386413999521,46cyclictest3868128-21date20:25:023
3864139995210,37cyclictest275-21dbus-daemon19:40:013
386413599524,8cyclictest3867906-21grep20:20:012
386413599522,14cyclictest3943076-21ssh23:45:262
386412599526,5cyclictest12-21ksoftirqd/022:35:250
386412599525,8cyclictest12-21ksoftirqd/022:50:340
386412599524,15cyclictest12-21ksoftirqd/022:15:390
3864125995226,11cyclictest3885995-21ssh21:43:480
386412599521,33cyclictest131rcu_preempt21:30:130
386413999511,46cyclictest3866638-21sshd19:59:073
386413999511,46cyclictest3866638-21sshd19:59:073
386413599512,6cyclictest1-21systemd20:10:012
386413599511,8cyclictest3866933-21modprobe20:05:002
3864130995132,14cyclictest3467602-21systemd-journal21:10:031
3864125995142,6cyclictest16950irq/224-800000021:15:000
386412599513,42cyclictest3945054-21unin-run23:50:000
3864125995116,31cyclictest12-21ksoftirqd/000:00:410
3864125995111,23cyclictest19350irq/34-mmc100:35:020
386413999501,46cyclictest150-21pr/ttyS219:12:123
386413999501,44cyclictest3866134-21modprobe19:49:073
386413999501,44cyclictest3866134-21modprobe19:49:073
386413599502,8cyclictest3867978-21modprobe20:25:002
386413599502,5cyclictest1-21systemd19:45:012
386413599502,11cyclictest3869478-21latency_hist20:50:022
386413599501,11cyclictest3868988-21sshd20:42:072
3864130995035,10cyclictest3467602-21systemd-journal20:14:591
3864125995037,10cyclictest12-21ksoftirqd/021:05:010
3864125995019,8cyclictest12-21ksoftirqd/000:15:010
386413999491,44cyclictest150-21pr/ttyS220:19:033
386413999491,44cyclictest150-21pr/ttyS220:03:333
386413599493,9cyclictest3864865-21munin-run19:25:012
386413599492,14cyclictest3864595-21modprobe19:19:592
386413599491,13cyclictest364-21snmpd20:56:192
3864130994934,12cyclictest275-21dbus-daemon20:05:011
3864130994933,10cyclictest3949750-21tr00:00:001
3864125994940,6cyclictest12-21ksoftirqd/021:46:340
3864125994934,12cyclictest1-21systemd20:35:000
386413999483,41cyclictest3868704-21modprobe20:36:073
386413999483,41cyclictest364-21snmpd20:49:103
386412599486,8cyclictest12-21ksoftirqd/021:51:280
3864125994811,33cyclictest151rcuc/020:20:010
386413999471,44cyclictest150-21pr/ttyS221:06:533
386413999471,43cyclictest364-21snmpd20:10:183
386413599472,5cyclictest3864170-21latency_hist19:10:022
386413599472,15cyclictest3867509-21latency_hist20:15:022
386413099474,40cyclictest3923906-21rm23:05:011
3864130994732,11cyclictest3874955-21cat21:20:011
3864130994732,11cyclictest3467602-21systemd-journal21:29:591
386413099472,3cyclictest131rcu_preempt22:34:451
386413099472,3cyclictest131rcu_preempt22:34:451
386412599478,35cyclictest151rcuc/019:20:010
3864139994616,25cyclictest3864570-21cron19:20:003
3864135994617,25cyclictest3467562-21sshd20:32:292
3864130994628,12cyclictest275-21dbus-daemon20:00:011
3864130994628,12cyclictest275-21dbus-daemon20:00:011
3864130994625,14cyclictest275-21dbus-daemon23:10:011
3864139994516,25cyclictest3864201-21latency_hist19:10:023
386413099453,3cyclictest131rcu_preempt00:29:391
3864130994532,9cyclictest309-21in:imuxsock19:52:071
386413099451,3cyclictest131rcu_preempt21:44:521
386412599458,7cyclictest12-21ksoftirqd/019:25:000
386412599455,6cyclictest12-21ksoftirqd/019:30:000
386412599451,8cyclictest3868302-21latency_hist20:30:020
386413099443,34cyclictest3933259-21ssh23:25:011
3864130994430,11cyclictest3954504-21sed00:10:021
3864130994430,11cyclictest23-21ksoftirqd/123:39:421
3864130994430,11cyclictest23-21ksoftirqd/122:00:001
3864130994430,10cyclictest275-21dbus-daemon20:20:001
386412599444,7cyclictest12-21ksoftirqd/020:45:020
3864125994433,7cyclictest12-21ksoftirqd/021:00:000
3864130994332,8cyclictest3928676-21sed23:15:021
3864130994328,11cyclictest23-21ksoftirqd/100:20:051
3864130994326,13cyclictest23-21ksoftirqd/122:04:241
3864130994326,11cyclictest275-21dbus-daemon19:25:011
3864125994336,4cyclictest16950irq/224-800000019:47:500
3864125994336,4cyclictest16950irq/224-800000019:47:490
3864125994335,5cyclictest151rcuc/020:40:000
386413099424,3cyclictest150-21pr/ttyS220:10:001
3864130994228,11cyclictest23-21ksoftirqd/100:40:001
3864130994225,13cyclictest0-21swapper/123:50:021
386412599427,6cyclictest131rcu_preempt19:55:020
3864125994215,23cyclictest12-21ksoftirqd/020:51:190
3864130994129,9cyclictest309-21in:imuxsock19:12:071
3864130994126,11cyclictest23-21ksoftirqd/100:19:571
3864130994122,13cyclictest3947358-21cron23:55:011
3864130994122,13cyclictest3947358-21cron23:55:001
3864130994121,15cyclictest3467602-21systemd-journal21:35:001
386412599417,5cyclictest151rcuc/020:15:020
3864125994134,4cyclictest16950irq/224-800000020:25:010
386412599412,9cyclictest12-21ksoftirqd/019:10:020
386412599412,5cyclictest3864118-21cyclictest20:10:010
3864125994110,8cyclictest12-21ksoftirqd/019:30:150
3864130994029,9cyclictest23-21ksoftirqd/121:40:021
3864130994028,5cyclictest3952204-21latency_hist00:05:021
3864130994026,11cyclictest23-21ksoftirqd/123:43:051
3864130994026,11cyclictest23-21ksoftirqd/123:43:041
3864130994025,11cyclictest23-21ksoftirqd/100:30:191
3864130994024,10cyclictest3467602-21systemd-journal22:55:001
386413099393,4cyclictest131rcu_preempt21:20:241
386413099393,4cyclictest131rcu_preempt21:20:241
3864130993929,6cyclictest3902343-21ssh22:19:061
3864130993927,9cyclictest23-21ksoftirqd/122:42:071
3864130993927,9cyclictest23-21ksoftirqd/122:08:011
3864130993925,11cyclictest23-21ksoftirqd/122:10:251
3864130993923,12cyclictest23-21ksoftirqd/122:35:541
3864125993932,4cyclictest0-21swapper/019:45:010
386413099385,23cyclictest3870297-21modprobe21:10:011
3864130993827,8cyclictest23-21ksoftirqd/123:34:191
3864130993827,8cyclictest23-21ksoftirqd/123:34:191
3864130993827,6cyclictest3868726-21cron20:40:001
3864130993826,9cyclictest3869490-21sshd20:52:071
3864130993826,8cyclictest23-21ksoftirqd/121:50:011
3864130993825,10cyclictest23-21ksoftirqd/119:40:031
386413099382,4cyclictest131rcu_preempt22:45:311
386413099382,4cyclictest131rcu_preempt21:05:011
3864125993829,6cyclictest151rcuc/020:02:470
3864125993817,16cyclictest275-21dbus-daemon20:50:000
3864130993724,9cyclictest309-21in:imuxsock19:35:021
3864130993724,11cyclictest23-21ksoftirqd/100:12:591
386413099363,3cyclictest131rcu_preempt23:28:411
386413099363,3cyclictest131rcu_preempt22:28:071
3864130993628,5cyclictest3864560-21sshd19:18:071
3864130993619,12cyclictest1-21systemd20:35:011
386413099361,4cyclictest131rcu_preempt22:20:441
3864130993528,4cyclictest0-21swapper/119:10:021
3864130993521,11cyclictest23-21ksoftirqd/120:30:011
3864130993517,14cyclictest1-21systemd21:55:011
3864130993423,9cyclictest23-21ksoftirqd/120:45:011
3864130993422,9cyclictest3869748-21sshd20:58:071
3864130993324,5cyclictest3866144-21sh19:50:001
3864130993324,5cyclictest3866144-21sh19:49:591
3864130993313,14cyclictest3467602-21systemd-journal19:40:011
3864130993122,7cyclictest0-21swapper/120:50:021
3864130993122,6cyclictest23-21ksoftirqd/120:24:421
3864130993121,7cyclictest23-21ksoftirqd/119:30:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional